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PREFACE 


Issues of flight deck automation are multi-faceted and complex. The rapid 
introduction of advanced computer based technology onto the flight deck of 
transport category aircraft has had considerable impact on both aircraft operations 
and the flight crew. As part of NASA’s responsibility to facilitate an active 
exchange of ideas and information among members of the aviation community, and 
since the timing appeared appropriate for a discussion of the effects of these 
changes on the roles of the crew and the automation, a NASA/FAA/Industry 
workshop devoted to flight deck automation was organized by the Aerospace 
Human Factors Research Division of NASA-Ames Research Center. 

The workshop was held at the Carmel Valley Inn in Carmel Valley, California, 
August 1-4, 1988. Participants were invited from industry and government 
organizations responsible for design, certification, operation, and accident 
investigation of transport category, automated aircraft. Attendees meluded 
representatives from airframe manufacturers, air carriers, the NTSB, the FAA, 
NASA and the university community. 

The workshop took a broad systems level perspective and discussed the design, 
training and procedural aspects of flight deck automation, as well as the crew’s 
ability to interact and perform effectively with the new technology. 

The goals of the workshop were to clarify the implications of automation, both 
positive and negative, and to identify issues regarding the design, training and 
operational use of flight deck automation. Participants with operational, training, 
or design experience were crucial to achieving these goals. 

A small preliminary, NASA workshop attended by NASA-Ames research staff and 
three human factors specialists was held to prepare for the industry-wide 
workshop. Thereafter, a source document on automation philosophies was 
prepared, and that report is included in the appendix. It is intended as an 
introduction to the concepts and ideas regarding flight deck automation which were 
discussed at this workshop. 

The remainder of this report presents the final results of the August workshop. The 
ideas and concepts in this document were developed by the workshop participants 
and written primarily by the NASA-Ames research staff. Since the workshop was 
small and informal, this report is not a verbatim transcript of the proceedings. 
Instead, the findings have been synthesized, where necessary, into an aggregated 
format and individuals and organizations have occasionally been de-identified. This 
format was chosen to facilitate a free exchange of information at the workshop. 



The workshop consisted of four major sessions: 

Introductory panels : Informal presentations by industry and 
government representatives on the design, operation, and certification 
of the automated cockpit. 

Formal papers: Formal presentations by members of the aviation 
community pertaining to automation in an operational context. 

Workshop group discussions : Six working groups were convened to 
discuss specific aspects of automation. These were: 

• Automation and Air-Ground Communication 

• Crew Coordination 

• Understanding Automated Systems 

• Training for Advanced Technology Aircraft 

• Error Management 

• Design Philosophies and Certification Issues 

Summary session : Summary and concluding remarks by the chair and 
other participants. 
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FLIGHT DECK AUTOMATION: 
PROMISES AND REALITIES 




INTRODUCTION 


Susan D. Norman 
Chair 

In developing the format and content for this workshop, considerable attention 
and time were given to the relationship between the aircraft design, certification 
criteria, operational procedures and training for transport category aircraft. 
Extensive discussions with aviation and human factors specialists were held at 
NASA-Ames, NASA-Langley and at various group meetings, including the ATA 
task force on human factors. The chair is much indebted to these people and 
discussions with them concerning the technical content and format of the 
workshop. 

In these discussions, it became clear that one important aspect of cockpit 
automation was the increasingly complex interaction between the processes of 
design, training, operations, and certification of the aircraft. The complexity of 
the new technology necessitated a clear understanding of the effect these processes 
have on one another in an operational setting. This was particularly important 
because, in time constrained situations, automated systems are often not as 
flexible as their human counterparts nor are their designs normally able to 
function as effectively as a human in nonstandard conditions. 

With this in mind, the panels and working groups were structured to explore the 
interdisciplinary, system level aspects of the automated cockpit. This workshop 
was to be a unique opportunity to understand the implications of how one element 
of the process impacted another; for example, how the implementation of a 
design or philosophy could impact the operational procedures or the interface 
with ATC. 

In preliminary discussions, it was also clear that the role of the pilot and crew 
was a key factor. Had it changed and if so, how? Was any apparent change 
moving in an appropriate direction? As a humorous exaggeration of the potential 
for a new role for the pilot, the question was asked if participants knew what the 
air transport crew of the 21st Century would look like? The answer was that the 
crew would be composed of two members, a pilot and a dog. The pilot was 
responsible for feeding and caring for the dog, and the dog was there to bite the 
pilot if he touched anything. 

Although certainly not intended to be realistic, this joke drives home the point 
about the potential role of the flight crew in future aircraft. Of the many 
unanswered questions presented to this workshop, those regarding the role of the 

5 


PRECEDING PAGE BLANK NOT FILMED 



crew were the most difficult. Is the potential for change in the role of the crew a 
valid concern?” and, if so, “Is this the direction the industry, as a whole, wishes 
to go?” 

Finally, it was intended that the workshop focus on operational realism, because 
real, versus perceived, problems and benefits need to be specified. Although there 
was considerable discussion of the theoretical aspects of cockpit automation, 
including philosophies of automation, the workshop was not intended to be 
theoretical in nature. It is important to understand and assess the existing situation 
before any changes, future research programs or philosophies are developed. A 
view toward the future is important, but a critical understanding of the current 
situation must form the basis for any discussions of the future. 
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Panel 

CURRENT FLIGHT DECK AUTOMATION: 
AIRFRAME MANUFACTURING AND FAA CERTIFICATION 

Prepared by Susan Norman and Kathy Abbott 

Moderator: Sam Morello, NASA-Langley Research Center 

Members: Harty Stoll, Boeing Commercial Airplane Company 

John Miller, Douglas Aircraft Company 
Don Armstrong, FAA Aircraft Certification 


INTRODUCTION 

The new generation of automated aircraft has increasingly used technology on the 
flight deck to enhance factors such as safety of flight and economic performance. 
The process of developing a new aircraft begins with the design where many 
fundamental and irrevocable decisions are made. Therefore, a discussion of 
automated aircraft technology should begin with the philosophy of design. 

The panel members were asked to describe, based on their extensive experience, the 
current philosophy of automation and to cite examples which illustrate how this 
philosophy has been implemented. The interrelationship of the certification process 
and its impact on the design were also discussed on this panel, since the FAA 
regulations must form the basis for an overall, system wide check on the 
performance of the aircraft in an operational environment. 

The major points from this session have been summarized and reorganized by topic 
to capture the essence of the presentations made by the panel members. The design 
and certification issues are discussed separately. 

DESIGN IN AN AUTOMATED ENVIRONMENT 

The manufacturers presented their perspectives with examples from the most 
recently designed aircraft. The figures were presented by Mr. Harty Stoll of the 
Boeing Commercial Airplane Company. 

Benefits of Automation 

It is important to state that automation has been a clear and continued benefit in 
terms of safety of flight. There is no question that the advances made in the 
reduction of the accident rate associated with human error have occurred to some 
extent because of the introduction of automation onto the flight deck. But, because 
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the workshop focussed on what can be done to improve safety of flight, most of the 
discussions centered on issues and problems as opposed to benefits. This is not 
intended to be a reflection on the technology itself. 


WORLDWIDE 737 OPERATORS DECEMBER 31, 1987 
OVER 200 OPERATORS TWO PILOT CREW tl.4B0 AIRPLANES) 25 MILLION FLIGHT HOURS 


ABU DHABI 
AER LING US 
AEROLINEAS ARG 
AERO TOURS 
AIR ALGERIE 
AIR ATLANTA 
AIR BELGIUM 
AIR BERLIN 
AIR CALIFORNIA 
AIR COMORKS 
AIR DJIBOUTI 
AIR EUROPE 
AIR FRANCE 
AIR FLORIDA 
AIR GABON 
AIR GUINBE 
AIR HAITI 
AIR LANKA 
AIR LIBERIA 
AIR MADAGASCAR 
AIR MALTA 
AIR MALI 
AIR NAURU 
AIR NEW ZEALAND 
AIR TANZANIA 
AIR ZAIRE 
AIR ZIMBABWE 
AIRWAYS 1NTL 
ALASKA AIRLINES 
ALL NIPPON 
ALOHA AIRLINES 
AMERICAN AIRLINES 
AMERICA WEST 
ANGOLA AIRLINES 
ANSETT AIRLINES 
ARAB INTL AIRLINES 
ARAMCO 

AUSTRALIAN A.F. 
AUSTRIAN AIRLINES 
AVIANCA 


BAHAMASAIR 
BAVARIA 
BEIJING 
BRAATHENS 
BRAN1FF 
BRAZIL 
BRITANNIA 
BRIT A1RTOURS 
BRITISH AIRWAYS 
BRITISH MIDLAND 
BUSY ?EE 
CAAC 

CAMEROON AIR 
CANADIAN AIRLINES 
CAYMAN AIRWAYS 
CONTINENTAL 
CHALLENGE 
CHINA AIRLINES 
CONDOR 
COPA-PANAMA 
CORSE AIR 
CP AIR 
CRUZEIRO 
DAN-AIR LONDON 
DELTA AIRLINES 
DOME PETROLEUM 
DRAGON AIR 
EAGLE AIR 
BG&G-LAS VEGAS 
BGYPT-GOVT 
EGYPTAIR 
EL AL 
ELDORADO 
EMIRATES AIRLINES 
ETHIOPIAN 
EURALA1R 
EUROPE AERO 
EXECUTIVE AIR 
FAR EASTERN AT 
FAUCETT 


FEDERAL EXPRESS 
FLUiUTPATH LTD 
FRONTIER 

GUANGZHOU REGION 

GULF AIR 

GUYANA AIRWAYS 

IIAPAG-LLOYD 

IUSFANIA 

IBERIA 

ILG TRAVEL 

INDIAN AIR FORCE 

INDIAN AIRLINES 

INDONESIA GOVT 

INTER EUROPEAN 

IRAN-GOVT 

IRANAIR 

IRAQI AIRWAYS 

ISRAEL INLAND 

ITEL AIR 

JAT 

KABO 

KOREAN AIR FORCE 
KUWAIT AIRWAYS 
LAUDA AIR 
L1NHAS 
LINA-CONGO 
LADECO 
LAN-CHILE 
LUFTHANSA 
LUXAIR 
MAERSK AIR 
MALAYSIA AIRLINES 
MARITIME 1NV CO 
MARK AIR 
MECOM CO 
MEXICO-GOVT 
MEY AIR 

MIDLAND MONTAGU 
MIDWAY AIR 
MIDWAY EXPRESS 


MONARCH AIRLINES 
NATIONAL AIRLINES 
NEW YORK AIRLINES 
N1GER-G0VT 
NIGERIA AIRWAYS 
N1SSHO-1WAI CO. 
NOGA 
NORDAIR 
NORDSTRESS 
ORION AIRWAYS 
PACIFIC EXPRESS 
PACIFIC SW 
PACIFIC WESTERN 
PAKISTAN INTL 
PAN AM 

PEOPLE EXPRESS 
PBTROLAIR 
PIEDMONT 
PLUNA 

POLYNESIAN AIR 

PRESIDENTIAL AIR 

QUEBECAIR 

ROTTERDAM AIR 

ROYAL AIR MAROC 

ROYAL BRUNEI AIR 

ROYAL DUTCH AIR 

ROYAL SWAZI 

SABENA 

SAHSA 

SAT 

SAUDI ARAB GOVT 

SAUDIA 

SAVAR 

SHARJAII-GOVT 
SINGAPORE AIR 
SKYBUS INC 
SOBELA1R 
SOUTH AFRICAN 
SOUTH WEST -JAPAN 
SOUTH WEST- US 


SPANTAX 
STAR JET CORP. 
SUDAN AIRWAYS 
SUN WORLD 
SUNLAND AIRLINES 
SURINAM AIRWAYS 
TACA INTL AIRLINES 
TAME-ECUADOR 
TAP/AIR PORTUGAL 
TEXAS AIR 
THAI AIR FORCE 
THAI AIRWAYS 
TRANS AIR 
TRANS EUROPEAN 
TRANSAVIA 
TRANSURASIL 
TRANSPACIFIC 
TUNIS AIR 
UNIT AR EM-GOVT 
UNITED AIRLINES 
UNITED AVIATION 
UNITED TECH 
UNIVERSAL 
US AIR FORCE 
US GOVT-NASA 
USAIR, INC, 

VARIG AIRLINES 
VASP 

VENEZUELA-GOVT 

VIVA 

WESTERN AIR 
WIEN AIR ALASKA 
XIAMEN 

YEMEN AIRWAYS 
YUNNAN 

ZAMBIA AIRWAYS 


THREE CREW (ONE AIRLINE-AIR FRANCE) 


Figure 1 


Although it is difficult to quantify the actual level of improvement in the accident 
rate associated with human error, there are some statistics which indicate this 
general trend. First, it is important to recognize the large number of flight 
operations and departures in today’s air transport system. Figure 1 lists the 
worldwide Boeing 737 operators as of December, 1987. It shows that there are 
about 200 air carriers and 25 million flight hours per year. 
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Second, the accident rate for crew caused accidents has been decreasing. Figure 2 
shows the data as a function of aircraft and its associated level of automation. 
Although there are many factors which impact these data, it is clear that the 
overall accident rate has not increased, and automation has been a factor m this 

reduction. 
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L NAV/V NAV AUTOPILOT 
FULL AUTO SUBSYSTEMS 
AUTO C/W READOUT 
QUIET/ DARK COCKPIT 


EFIS/EICAS 
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AUTO IGNITION 
WINDSHEAR ALERT 
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707 
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747 


737 

- 100/-200 


737 

- 300/-400 


757/767 


Figure 2 


The Development Process 


The design process for developing a new aircraft is complex. Improvements in 
the design to reduce the potential for human error are an active part of the design 
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process. Manufacturers consult flight test pilots and operational carriers to 
determine their needs and assess the impact of technology. 

Boeing has a formal Flight Deck Design Committee which reviews accident data 
and makes specific recommendations regarding the design of the flight deck. 
These new design options are available only because of the new automated or 
computerized technology which exists today. Examples of the accident related 
causes and associated design improvements which were incorporated into the 
Boeing 767/757 design are given in Figure 3. 

In addition to examining accident statistics, Boeing also reviews major problems 
and identifies their associated subproblems. A recommendation is then made for 
an improvement in the design. As examples. Figure 4 lists the major 
recommendations for the 747-400/737-300. 
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Boeing Flight Deck Design Committee 


Examples of Accident Data Reviewed 
Subsystem Management Accidents — Worldwide Air Carriers 

1968-1980 


Accident Related Cause 

• Crew omitted pitot heat 

• Wrong position of standby power 
switch 

• Flight engineer and captain con- 
ducted unauthorized trou- 
bleshooting 

• Electrical power switching not co- 
ordinated with pilots 

• Flight engineer shut off ground 
proximity warning system 

• Faulty fuel management 


• No leading edge flaps on takeoff 
with digital computer 

• Confusion over correct spoiler 
position 

• Crewman did not follow pilot’s 
instruction 

• Mismanaged cabin pressure 


767/757 De sign Improvements 

• Auto on with engine start 

• Automated standby and essential 
power 

• Simplified systems; delete 
maintenance functions 


• Auto switching and load shed- 
ding — no crew action required 

• Shut off on forward panel in 
full view of both pilots 

• Auto fuel management with alert 
for low fuel, wrong configuration, 
and imbalance 

• Improved takeoff warning 

• Dual electric spoiler control 

• Full-time caution and warning 
system 

• Dual auto system with auto 
switchover 


Figure 3 
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Boeing Flight Deck Development Committee 
Major Problems Identified 


Problem 

1 ) Lack of timely aircraft 
position information 


2 ) Engine control & moni- 
toring caused high 
workload 


3 ) Inadequate caution and 
warning system 


4 ) System management 
causes high workload 
and errors 


5 ) Inadequate design evalu- 
ation methods 


6) Display reliability 


Sub-Problem 

• Approach information 
difficult to read 

• Exact airplane positions 
not always known — high 
workload 

• Difficult to plan ahead 

• Difficult to rapidly set 
desired thrust 

• Large number of throttle 
adjustments 

• Lack of alerting for out- 
of-tolerance condition 

*> Excessive aural and nui- 

sance alerts 

• Lack of standard color 
usage 

• Alerts not centralized or 
categorized 

• Increase of hardware and 
functions of systems in- 
creases error 

• Monitoring of out-of- 
tolerance conditions in- 
creases workload 

• Increase procedures 

• No means to evaluate de- 
sign before they are in- 
tegrated in cockpit 

• Single systems limit er- 
ror detection 

• Panel temperature reduces 
reliability 


Recommendation 

MAP display 
FMS 

INS-type information 


FMS, MAP plan mode 
Electronic engine control 


Alerting means included 
with engine instruments 

Quiet, dark flight deck 


Standardize colors 


Central alert with im- 
proved logic — simplified 
systems 

Simplify systems 
Simplify panel hardware 


Add redundancy and au- 
tomation so no immedi- 
ate crew action required 


Develop new computer- 
ized workload techniques 


Digital computers 
CRT displays 

Liquid crystal display 
Reduce LRUs 
New concept for equip- 
ment cooling 


Figure 4 
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7 ) Irritation and crew fa- 
tigue items 


8) Inadequate or uneven Il- 
lumination 


9) Poor readability of dls 
plays 


10) Inadequate landing vi- 
sion and vision for col- 
lision avoidance. 


Cockpit noise 

Air conditioning and cir- 
culation 

Gye fatigue 

Seat comfort 

Inadequate stowage 

Instruments and lighted 
pushbuttons too hot 

New displays make light- 
ing balance more diffi- 
cult 

Improved contrast 

Visual angle too small 

Low minimums require 
better down vision 

Windows not designed to 
collision avoidance 


Figure 4 (continued) 


New equipment and re- 
quirements 


New pushbutton concept 
required 

Automatic dimming con- 
trols 


New equipment to higher 
standards 


• New window design 
criteria 
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Design Philosophies 


The Boeing design philosophy is best characterized by an emphasis on simplicity 
first, then redundancy, and last automation (Figure 5). 


EFFECTIVE SYSTEMS DESIGN: 


Simplicity 

Redundancy 

Automation 


Figure 5 


As examples of the implementation of this philosophy, the design of the Boeing 
747-400 has centralized and reduced the crew alerting system so that it is simpler 
for the crew to determine what is happening. Another example is the fuel system 
on the 747-400. The original design specified 5 fuel tanks per wing because of 
structural considerations in the wing. But, from an operational standpoint, this 
design needed a total of 10 boost pumps (two per tank). The resulting fuel 
crossfeed procedures were determined to be overly complex and the design team 
elected to simplify the overall system by implementing a baffled tank structure. 
The resulting system had only 3 tanks per wing and used some automation in the 
middle tank to assist in the reduction of operational complexity. This illustrates 
the importance of first designing to simplify which may, in turn, reduce the need 
to automate. 

The Boeing automation philosophy is summarized in Figure 6. Automation does 
have a vital role in aircraft safety design. This is best exemplified in the 
automation of engine controls. With the introduction of appropriate automation, 
the engines can now be fire-walled simply by moving the throttles forward and it 
is not necessary to adjust any other controls. 

Another important issue for the design of state-of-the-art aircraft is the pilot role. 
John Miller of Douglas indicated that the MD-11 has been designed with the 
criteria that any irrevocable action, such as engine shutdown, must have a manual 
pilot action. 
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Automation Philosophy 


• Simplified/Minimized Crew Procedures for Subsystem Operation 

• Reduces random and systematic error 

• Increases time for primary pilot functions 

• Prevents requiring any immediate crew action 

• Reduces subsystem mismanagement accidents 

• Centralizes crew alerting for error reduction 

• Allows “fire walling” engine controls 

• Allows two member crew operation 

• Improved Navigation Information 

• More exact airplane position indication (MAP) 

• Reduces fuel usage 

• Higher reliability and improves accuracy 

• Reduces crew error 

• Reduces workload, allows more pre-planning 

• Improved guidance and control 

• Reduces workload 

• Allows low minimum operation 

• Allows manual, semi-automatic or automatic pilot flight 

• Increases precise guidance information 


Figure 6 


Both manufacturers’ design philosophies included the concept that the pilot should 
primarily be responsible for flying the aircraft. A minimum of crew procedures 
facilitated this task. For example, during the MD-11 design process, procedures 
were minimized where possible, particularly when the procedure could be pre- 
specified (i.e., there were no options or choices). For example, the MD-11 has six 
CRTs, and in the case of a failure of one, there is a specific, recommended 
reconfiguration for the five remaining displays; similarly with 2 failures, etc. 
Instead of making this reconfiguration process a procedure, the decision was 
made to automate it, because the reconfiguration could be predetermined. The 
crew would then be able to focus their attention on flying the aircraft, instead of 
referring to a procedural manual to reconfigure the CRT displays. 

Regarding function allocation, both manufacturers indicated that the subsystem 
management area was most amenable to automation because of the highly 
proceduralized nature of the task. The Boeing philosophy of allocation is 
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illustrated in Figure 7. Those tasks associated with the guidance and control of the 
aircraft have remained with the most direct crew involvement because these tasks 
are dynamic and critical. 


Functions Allocated to Crew 


• Guidance 

• Control 

• Separation 

• Navigation 

• Systems Operation 



INCREASING 

CREW 

INVOLVEMENT 



Figure 7 


Lessons Learned 

With the introduction of systems such as the CDU, a lot of heads-down time 
resulted, especially at first. During the original design, such systems were not 
intended to be used during periods of frequent ATC changes, particularly in the 
terminal area. However, a great deal of emphasis was placed on the FMC/CDU 
during the certification because it was a new system. It was also emphasized 
during training, which may have caused an over-emphasis during check rides. 
The result was that pilots may have felt obligated to use it in the terminal area 
even though this was not the original design intent. 
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CERTIFICATION IN AN AUTOMATED ENVIRONMENT 


A brief overview of the FAA certification process was presented by Don 
Armstrong. There are three basic phases: setting standards, substantiating designs 
by analyses and laboratory tests, and flight tests. 

With respect to the standards, the mles attempt to be as objective as possible. For 
automatic digital systems, there are three basic requirements: 

1) The system must perform the intended function. 

2) The system must meet all applicable requirements; e.g., 
specific rules governing electrical loads, circuit protection, 
electromagnetic interference, flammability, etc. 

3) The system must interface properly with all other systems, and 
with die crew. 

However, new concepts often do not have suitable standards, and it becomes 
necessary to work with the applicant to determine appropriate certification 
criteria. In fact, the rules are inherently two to three aircraft generations behind 
the state of the art of the technology to which they apply. 

The next aspect of certification involves design substantiation, which involves 
three aspects for automatic and/or digital systems: software testing, hardware 
testing and the integrated system level tests. Software is categorized into three 
classes according to the importance or criticality of failures: critical, essential, 
and non-essential. Software which is critical to continued safe flight and landing 
(where failures can have potentially catastrophic results) is classified as critical. 
The most rigorous analyses and tests are conducted to assure that such failures 
will be improbable or extremely improbable. Equipment whose failures result in 
little effect on continued safe flight is classed non-essential, and their failure rates 
can therefore be relatively high. 

Hardware is given the usual “shake and bake” testing which has been the industry 
standard for years. The hardware and software are then tested in an integrated 
manner to assure that the components function as expected. 

The final phase of the certification process is flight testing. Every major sequence 
of events (changes from one mode of operation to another) is included in the 
flight scenarios. However, since it is not possible to evaluate every conceivable 
situation or series of mode changes, the flight tests must inevitably be spot checks 
of families of logical sequences. Cockpit layouts, including labeling, lighting, and 
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annunciation under normal and abnormal conditions, are evaluated. Finally, 
workload effects are assessed, but, in the end, the final flight test results represent 
the consensus of the subjective evaluations of the flight test pilots. 

This workshop offered an opportunity to air some concerns regarding the current 
certification process in the hope that a broader perspective could be gained and 
that certain issues might be further clarified. Some of these issues are: 

1) The current rules do not have any comprehensive human factors 
requirements; instead, the subjective evaluation of the flight test 
pilot determines certifiability. This is worrisome not because of the 
subjective evaluation itself, but because the design engineers have 
made critical design decisions long before the flight tests occur, and 
changes are routinely resisted due to cost and schedule impacts. In 
the absence of rules which incorporate human factors criteria, the 
question is whether these design decisions adequately reflect the 
concerns of the FAA test pilots and the line pilots they try to 
represent. 

2) Although this is difficult to admit, modifications by a manufacturer 
to previously approved models, or installations of new systems to 
existing aircraft by modifiers resulting in STCs*, are not usually 
rigorously evaluated for human performance criteria such as crew 
workload. This is becoming an increasingly important issue because 
of the increased complexity of these modifications, particularly in 
the business jets. 

3) It is difficult to focus on standardization since the dominant 
manufacturing strategy is to meet the customer’s requirements. For 
example, it is possible to present information using different 
colors, signals on or off, aural messages which are loud or soft, 
high numbers up or down, data which are always present or 
sometimes inhibited, etc., etc. The situation is further complicated 
by the fact that the industry has diverse opinions. These opinions 
even differ within the same manufacturer, let alone between the 
various FAA offices, and even perhaps between the test pilots 
within a single office. There are many cries for standardization, 
but the FAA is understandably reluctant to impose its will because 
to do so not only dictates design, but may also inhibit innovation. 


*Supplemental Type Certificates 
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4) Flight test pilots of necessity make individual subjective 
assessments. The primary concern here is that the lack of standards 
for such assessments may not allow predictable results to be 
achieved among the small universe of pilots in the several FAA 
offices. 

5) There is a critical need for a definitive, widely accepted training 
course in human factors for FAA test pilots. 

Tn conclusion, a quote from St. Paul in his Epistle to the Romans was cited: You 
wish to have no fear of the authorities? Then continue to do right and you will 
have their approval, for they are God’s agents working for your good.” 
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Panel 

REALITIES OF OPERATING AUTOMATED AIRCRAFT: 

AIR CARRIERS AND FAA- AIRCRAFT EVALUATION 

Prepared by Susan Norman and George Steinmetz 

Moderator: Earl Wiener, University of Miami 

Members: A1 Ogden, United Airlines 

Jack Wisely, TWA 
Stu Grieve, Britannia Airways, Ltd. 

Rod Lalley, FAA-Aircraft Evaluation 

INTRODUCTION 

The previous panel discussed the design and certification processes. In the real 
operational world, however, unforeseen events occur which can cause problems 
even with the most thoroughly planned and carefully engineered systems. With this 
reality in mind, the representatives of the air carriers and an FAA aircraft 
operational evaluation organization were asked to discuss issues associated with the 
introduction of the new generation of automated aircraft and to cite examples, 
based on their operational experience, which illustrated these issues. 

This paper summarizes the panel presentations and discussions, but is not intended 
to be a consensus of the panel members. It is organized by major topics which may 
overlap in content because automation, itself, is a highly complex subject. Since 
most of the important points made during the panel discussion are related to the 
technology and not the carrier or manufacturer, references to specific air carriers 
and manufacturers have, for the most part, been deleted. 

LESSONS LEARNED FROM OPERATIONAL INCIDENTS 

Benefits 


The benefits associated with autoflight are substantial and this fact should not be lost 
even though most of the panel discussion, and hence the report, concerned how to 
improve our use of automation. As an example, the Boeing 767 was one of the first 
aircraft to employ substantial automated technology. United has been flying this 
aircraft since 1982, and they have never had an accident or incident resulting in 
damage to the aircraft. This, in part, is a result of the extremely good engineering 
of the aircraft. There has been only one incident requiring an NTSB investigation 
which was later discontinued. 
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A specific example of one of the benefits of automation is the fact that United has 
never had a reported altitude bust on the Boeing 767 when the altitude was 
correctly set. 


Operational Crutches 

One of the lessons learned regarding new technology aircraft is that we should 
not build operational crutches to get around improper designs. An example was 
given to illustrate the importance of this lesson. During the early days on the 767, 
a problem occurred (which has now been fixed) with the electronic fuel control. 
An over-sensitivity in the engine EGT (exhaust gas temperature) sensors 
occasionally caused spikes in the EGT data which, in turn, could lead to an 
automatic down trim of the engine. This was a potentially serious problem, 
particularly during takeoff. The engine manufacturer promised to reprogram the 
engine control software, but the air carrier management felt that a temporary fix 
or ‘operational crutch’ was necessary until the permanent fix could be 
implemented. A new operational procedure was instituted which specified that the 
EEC (electronic engine control) should be turned off during takeoff. However, in 
turning it back on shortly after takeoff, a crew inadvertently manipulated the fuel 
control switches, which were located close to the EEC switch. This resulted in the 
shut down of both engines, which were soon restarted. 

In this example, a human engineering design, a management decision and a fast 
action on the fuel control switch all came together to contribute to a serious 
incident. 


Training/Operational Procedures 

In training programs, it is important to remember that the pilot is flying an 
aircraft, not a computer. Due to the novelty of the technology, some early 
training programs emphasized the automatic and computer aspects of the aircraft 
with some loss of emphasis on basic airmanship. As a result, instruction in 
fundamental flying knowledge, such as altitude and power settings to effect 
different configurations, may have suffered. More recent training programs 
emphasize basic manual flying skills, including appropriate flap and power 
settings. 

At one air carrier, some situations occurred during the 1970s on the 727 which 
emphasized the need to develop a crew procedure for every potential failure. The 
resulting training conditioned the pilot to expect that there is a procedure to 
follow for every failure. This operational philosophy worked well before the 
introduction of the Boeing 767 but did not work well on this aircraft. This is in 
part due to the fact that formulating procedural solutions for automated 
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technologies in advance can be difficult because of the sheer complexity of pre- 
determining all potential failures. But, fortunately, the 767 is very well 
engineered and no major problems resulted. One positive way to improve this 
situation is to teach pilots an understanding of how die systems work together and 
how they impact each other. 

Understanding how the system works is further illustrated by a situation 
regarding holding patterns which occurred during the early operations of the 
767. Some pilots apparently did not understand how the flight management 
computer (FMC) drew the holding pattern. While they were busy inserting the 
holding pattern data into the FMC, one crew failed to remember to slow the 
aircraft down to the recommended holding pattern speed. This, coupled with a 
strong tail wind, resulted in an overshoot of the assigned pattern. 

A team approach to new aircraft training is important. One carrier’s 
representative indicated that simulation is absolutely essential and improved 
facilities are needed. 

When highly automated aircraft were initially introduced, one carrier had 
concerns about older pilots and their ability to adjust to the new technology. The 
older pilots, however, had little difficulty learning the new systems. The only 
difference was that the younger pilots learned more quickly. 

Requalification training should be very thorough and this is an area where 
improvement may be needed. For example, “de-programming the expectations 
of a pilot who is returning to a less automated aircraft may be necessary. A pilot 
transitioning from a highly automated aircraft to a less automatic one may 
unconsciously expect it to automatically level off at altitude, but some older 
technology is not capable of this. 

An important issue is mixed fleets where pilots fly both sophisticated and non- 
sophisticated aircraft. The issue is not the individual aircraft, but mixing aircraft 
designated as a common type which have differing design philosophies into one 
fleet. At least one carrier split its fleet due to concern about this issue. 

Mode Misapplication 

It is possible with the new computer technology for the crew to assume that the 
aircraft is operating under one control mode when, in fact, it is not. This is 
particularly important when default modes are involved. One example of a mode 
misapplication which resulted in a high altitude stall warning occurred on the 
767. The aircraft was climbing with the vertical speed mode engaged. In this 
mode, no automatic limits on vertical speed are designed into the system. This 
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means that the aircraft automation will never over-ride or limit the selected 
vertical speed even if a stall is imminent or the speed is too high. The fact that 
this is the default mode makes it particularly troublesome due to the potentially 
serious consequences. Such default modes should be avoided in future designs. 

Over Reliance on Automation/Electronic Map Failures 

Although rare, electronic maps have failed and crews should not be trained to be 
overly dependent on them. It is, however, easy to become accustomed to these 
map displays because of their usefulness and quality. In training, the use of charts 
needs to be re-emphasized as well as the need to maintain good situational 
awareness. 

When things do go wrong, there may be a reluctance by the crew to turn the 
automatics off. There is a tendency to try to use the automatics to cope with 
rapidly changing circumstances even if there is not enough time to enter the new 
data into the computer (Editor’s note: the term “reprogram” is commonly used 
for this process, but technically reprogramming involves modification of the 
system software, not data entry). Data entry into the FMC (flight management 
computer) can also be a procedural concern. The early FMCs were slow and it 
was easy to get ahead of the display. Pilots should be trained to check the 
correctness of the numbers before the “execute” button is pressed. 

ATC/Automated Aircraft Compatibility 

Current designs for the automated navigation systems interface very well with the 
existing ATC system as long as there are no flight plan changes. However, the 
reality is that changes (sometimes frequent) to the flight plan and hence the FMS 
(flight management system) are required for weather, traffic, enroute spacing 
slow downs, etc. There may be a tendency to teach airline crews that the 
automated systems can accommodate these changes. Unfortunately, the data entry 
may often take more time than the ATC environment allows. 

In addition, frequent ATC changes to the flight path can render the automatic 
system useless because the data take too long to enter, particularly below 10,000 
feet. For example, vertical navigation systems work well above 10,000 feet, but 
sometimes are not very workable below this altitude due to speed and altitude 
restrictions. Since these restrictions are Often deleted from the SIDs (standard 
instrument departures), changes are hard to keep up with. The design of the MD- 
11 may not have this problem since speeds can easily be changed 
(reprogrammed). The integration between the mode control panel and the FMC 
will also be a good feature of the MD-1 1 and 747-400. 
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Another example of the problems which can arise due to the mismatch between 
the new automated aircraft and ATC, involved a crew mis-manipulation of the 
mode control panel and an ATC inability to permit the aircraft to descend when 
requested. As a result of the inability to descend, the aircraft got further and 
further off of its planned descent path. In an attempt to recover the situation, the 
crew shut off the autopilot and recycled the computer. This fixed the pitch 
problem, but unfortunately the crew did not realize that, although they were 
tracking the localizer, the automatic capture had also been removed. This 
illustrates the need to improve the compatibility between ATC and the automated 
aircraft, as well as the need for crews to understand how the automatics work. 

The controllers need to understand the capability of the newer generation of 
aircraft. For example, an ATC controller may give a change in course or 
clearance and expect to see a change in the aircraft course displayed on the radar 
screen. But with a modem aircraft, a course change may not be immediate, 
because the crew is busy entering the new course data into the flight management 
computer and not executing the requested course change. 

As a last point, the national airspace must be viewed as a system. Many of the 
benefits of improvements in the cockpit cannot be realized without improvements 
in the ATC system as well. 

Computer Sensitivitv/Maintenance Issues 

There have been examples where an automatic system has caused movement of a 
control surface. In particular, some uncommanded flap extensions at altitude have 
occurred which were later determined to be a maintenance problem. Although 
these situations have resulted in increased workload for the crew, no serious 
situations have occurred. Computer over-sensitivity is a factor that should be 
considered during certification. 

An example of computer under-sensitivity occurred during a windshear condition 
on a commercial flight. The changes in pitch commanded by the autopilot caused 
the autothrottle to inappropriately change the engine thrust. The inability of these 
two systems to get together contributed to a 3000 foot per minute descent rate 
which was arrested at about 600 feet. The pilot had to take manual control of the 
aircraft and elected to execute a go around. No accident resulted. In summary, 
the inability of the autopilot to respond adequately to the rapidly changing 
meteorological conditions caused large pitch excursions close to the ground. 

Training and expertise of maintenance personnel are also factors. Many 
maintenance personnel have their primary experience with older technology 
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aircraft yet they are now responsible for maintaining the new, computerized 
systems. 


Design 

Primary flight displays should be structured to reduce clutter, as well as to be 
appropriate for the mode of flight. As indicated previously, electrical failures, 
although rare, do occur and the designs should maintain critical flight instruments 
under these conditions. 

Consideration should be given to improving the CDUs to reduce the current 
keying requirements . 1 In busy terminal areas, there should be minimal 
interaction with the CDU systems. The CDU does, however, invite crews to “play 
with it.” 


Soft Failures 


Soft failures are situations which were not anticipated by the manufacturer, so 
there are no pre-specified procedures, but something is clearly wrong with the 
cockpit display (i.e., the failure is not significant enough for the computer to 
indicate a failure). Such failures cause difficulties because of the nearly 
impossible task of predicting them. This, in turn, causes problems in preparing 
appropriate operational procedures. 

One carrier has had at least two documented cases of computer “soft” failures. In 
both cases, the display showed no electronic course line. The first time this 
happened, crew training had been oriented toward reliance on the electronic map 
and there were no pre-defined operational procedures. The crew was not exactly 
sure how the IRU (inertial reference unit) and the FMC were linked together. 
They landed the aircraft safely but were not able to correctly diagnose the 
problem. It was written up as a triple IRS failure which had, in fact, not 
happened since the display still had other navigational data. Changes were then 
made to the educational program and as a result, the second time this happened, 
the crew was better prepared for the situation. The captain simply selected the 
VOR mode and safely continued the flight. 

Basic Standards 

Basic standards regarding implementation of automation across aircraft are 
needed, particularly for default modes. For example, the normal default mode, 
when the autopilot is engaged, should be speed and pitch hold. In the example 


1 Editor's note: The Boeing 747-400 and MD-11 have improved this. 
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cited previously, where the default mode is the vertical speed hold, the situation 
would not be very comfortable with one engine out. 

Minimum Equipment List (MEL) 

We may do ourselves a disservice by the specification of equipment on the MEL 
which allows a degradation of the total system. One example is the need for an 
operational APU (auxiliary power unit). Under certain conditions, it is not 
required. Yet, this is clearly not an optimum situation if an engine failure should 
occur during a Category II landing. 


Workload 


Workload on the pilot not flying, particularly in a terminal area while the 
aircraft is being flown manually, can be very high. 

Britannia Airways Ltd. has used heart rate 2 data to augment subjective pilot 
ratings of workload. Captain Stu Grieve of Britannia showed data on heart rate 
of pilots under various conditions. Heart rate measurements were taken for crews 
flying the Boeing 767 and the 737 which have very different levels of 
automation. Both take-off and landing flight phases, as well as different operating 
modes, were measured. The difference between the 767 and 737 is illustrated in 
Figure 9 for similar ILS approaches at Luton using the flight director. The heart 
rate for the 767 approach is about 10 beats/minute lower than for the 737. 

Figure 10 compares the heart rate responses during standard instrument 
departures out of Luton. On the 767, the autopilot is engaged at about 500 feet 
before the aircraft is cleaned up. On the 737, due to noise abatement procedures, 
the autopilot is engaged after the flaps are retracted and the aircraft is in trim. 

As a last comparison, Figure 1 1 shows the difference in heart rates for different 
operating modes during a standard instrument departure from Luton in the 767. 
Compared to hand flying (bottom trace), heart rates are reduced when an 
autopilot (top trace) is used. Rates are also reduced when a flight director which 
is driven by the flight management system (FMS) is used (middle trace). 

In summary, for the take-off and approach to landing phase, the Boeing 737 
crews had generally higher heart rates than the 767 crews. However, the rates 
during the actual flare to touch down flight phase were approximately equal for 


2 Editor's note: Since heart rate can be affected by factors other than workload, data interpretation 
can be an issue. These data are, however, presented without interpretation. 
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both aircraft. These heart rates were also higher for actual flight conditions than 
would be expected in the simulator and this was probably due to an inability to 
properly simulate the real world, particularly wind conditions. 


Heart Rate 


(F/D FLS) 
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APPROACH and LANDINGS— LUTON 
Comparison of Heart Rate Responses for the B737 and B767 — Same Pilot 

Figure 9 

Heart Rate 



SIDS— LUTON 

Comparison of Heart Rate Responses for B737 and B767 


Figure 10 
30 







B767 SIDs LUTON 


Heart Rate 



B767 SIDs— LUTON 

Comparison of Heart Rates for Different Operating Modes — Same Pilot 

Figure 11 

CONCERNS FOR CERTIFICATION OF AUTOMATED AIRCRAFT 

The Airbus 320 accident in Europe which occurred in July, 1988, just prior to 
the workshop, was discussed. The information available at that time was, 
however, necessarily incomplete. Since more complete reports are now available, 
the accident discussion is not included here. Several important points regarding 
operational certification of advanced technology aircraft were, however, raised 
and these are described. 

Crew alerting : Crew alerting should be examined carefully during 
the certification process for automated aircraft. In the A320 
accident, apparently minimal crew alerting systems indicated the 
onset of an unsafe air speed. There were, apparently, no aural or 
tactile (stick shaker) warning devices. The only indication was the air 
speed indicator itself. A question, from the audience, was raised 
concerning aircraft designs which prevent a pilot from taking action 
instead of alerting him/her. Escape from windshear is another 
problem area. 
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Manual operation of an automated aircraft : Since the accident 
occurred during a manually controlled fly-by, the implications for 
operation of automated aircraft in the manual mode need to be 
carefully examined. Procedures for operating highly automated 
aircraft in the manual mode need to be fully understood. 

Crew experience requirements : As a general concern, less 
experienced pilots may be more likely to be assigned to aircraft like 
the A320 because of its smaller size. The impact, if any, of relatively 
limited flight crew experience on safety of flight is not known. 

Crew over-confidence in automation : Questions have been raised 
regarding the apparent over-confidence of crews regarding the 
capabilities of automation. In the A320 accident, the question was 
asked, “Would the crew have attempted such an extremely low, slow 
fly-by in a more conventional aircraft?” Although there is no way to 
answer this question, the consensus seemed to be “No.” It was also 
stated that the A320 is a very impressive aircraft and is capable of 
maneuvers that are not possible in conventional airplanes. 

Pilots switching between automated and less automated aircraft : The 
ability of crews to switch between automated and less automated 
aircraft in a safe manner remains a concern. If crews become 
accustomed to automatic protection features, will they be able to 
adjust to a less automated aircraft environment without considerable 
additional training? 

Circumstances where automatic protection is a clear benefit : The 
A3 20 accident may, however, prove to demonstrate the benefit of the 
automatic flight envelope protection features. Although preliminary 
information suggests that the alpha floor protection was inhibited, 
the aircraft appears to have remained stable as it descended into the 
trees. The angle of attack protection probably had the beneficial 
effect of lessening the severity of the impact. 


The concerns raised about automated aircraft certification are best summarized 
by the phrase “vulnerable systems, fallible humans.” The unfortunate A320 
accident was perhaps an example of one or both. 
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OPERATIONS SUMMARY 


In his summary, Capt. A1 Ogden of United stated that a combination of factors 
are frequently involved in incidents with automation. Some of these are: 

1) Inadequate operational knowledge. Lack of adequate operational 
knowledge can lead to a failure on the part of the crew to 
understand how to operate the systems. Training should 
emphasize the integration of the systems and how they work 
together. We tend to teach how the system was designed by the 
manufacturer, not how the systems are operated or integrated. 

We tend to teach “How does it work?” when we should be 
teaching “How do you work it?” 

2) Cockpit discipline. Allocation of responsibilities between the pilot 
not flying (PNF) and the pilot flying (PF) should be rigorous. For 
example, at United, the PF does not set the altitude window. The 
focus of communication in the cockpit is the mode control panel 
(MCP). United has been very successful with this approach for the 
MCP but less successful with the FMC/CDU. 

3) Cockpit resource management : Workload needs to be carefully 
controlled. Too many tasks can be placed on one person, 
particularly when data entry (“reprogramming”) is required. 

4) Split mode operation : Operation in split mode (i.e., operation of 
the autopilot without the autothrottle) is discouraged at United. 

For example, either all autoflight or all manual is encouraged. 

5) Switch discipline: Simply stated, this means know which switch 
you are going to move before you move it. 


Capt. Ogden also suggested the following list of needs: 

1) Better Technical Publications are needed particularly on “How 
do you work it?” Current training material explains very well 
how the systems work, but is not as good on how to work the 
automatics. 

2) Analysis before action should be stressed and peripheral 
distractions should be eliminated before beginning the analysis 
process. 


33 



3) Tighter standardization of operational procedures is needed. 

4) Better definition of tasks and priorities would be helpful. 

In summary, automation is a definite plus which has many benefits including 
workload reduction, but it is not a panacea. Automation must work for us, and 
not the other way around. 
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Invited Paper 

FIELD STUDIES IN AUTOMATION 

Dr. Earl Wiener, University of Miami 


Susan Norman: Dr. Wiener has been conducting a field study with several major 
carriers regarding the introduction of automated aircraft. Although the study is not 
yet complete, Dr. Wiener requested and was granted permission from the 
cooperating carriers to present the interim findings. Part of the success of the 
workshop was due in part to the diversity of opinion presented. Dr. Wiener was 
asked, as an unbiased observer, to candidly raise some of the more critical issues 
regarding the pilots’ operational perspective of automation. His findings and the 
resulting discussion are reported here. 


Dr. Wiener: 

This afternoon. I’d like to present an interim report on a field study I am 
conducting on the 757 in cooperation with two large airlines. Field studies are a 
window on the real world. The ASRS database is another window on the real world. 
By looking through these and other windows, we can leam important things about 
the operation of the world. The title of this study, shown in Figure 1, involves error 
analysis, but error analysis is just one of its features. 


ERROR ANALYSIS AND PREVENTION 
IN HIGHLY AUTOMATED AIRCRAFT 

A FIELD STUDY OF 757 CREWS 


EARL L. WIENER, PRINCIPAL INVESTIGATOR 
EVERETT PALMER, CONTRACT TECHNICAL MONITOR 


Figure 1 
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A field study brings together a large number of very diverse groups. The first job is 
to actually bring them together. I sincerely appreciate the whole-hearted support 
I’ve gotten from the two airlines and from die safety committees of ALPA. I 
particularly appreciate it because both of these airlines in the last few years have 
had acquisitions and other major challenges. It would have been very easy for them 
to keep a proposed research project on the shelf. 

As a historical perspective, in 1979, I came to NASA-Ames to work with Ren 
Curry on a new automation project. We spent a year trying to figure out what the 
automation project should be. I felt it would be interesting to get out into the field 
where real people were doing real jobs and see what was going on. Ren launched a 
study on the Boeing 767 and we agreed our main interest was in crew transition. We 
wanted to see what happened in those early months when people transition from 
traditional aircraft to those with new technology. 

That study was done at Republic Airlines. It was ideal for what I wanted to study. 
They had experienced, traditional DC-9 pilots, one of the largest DC-9 fleets and 
were transitioning into MD-80s. The MD-80 was the first of what I considered 
modem aircraft in the Republic fleet. So except for whatever military or corporate 
experience their pilots might have had, the DC-9 represented their most advanced 
technology. Furthermore, since they were transitioning from a DC-9, the 2 crew 
versus 3 controversy would not be a factor in their transition to this new advanced 
cockpit technology aircraft. 


AUTOMATION FIELD 

STUDIES 

Wiener (1985) 

MD-80 

Crew Transition 

Curry (1985) 

B-767 

Crew Transition 

Wiener (1988) 

B-757 

Error Analysis 
Crew Coordination 



Training 

Workload 


Figure 2 
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My present study on the 757 seeks to extend beyond just crew transition into flight 
crew errors in advanced technology. I’m interested in crew coordination 
particularly in the modem aircraft. Training, which is an obviously important 
factor, has long been of interest to human factors investigators. Finally I want to 
talk about workload and workload management. 


WHY DO FIELD STUDIES? 

• Realism of the operational world 

• Line crews are untapped source of data 

• Problems often do not appear until years of line 
experience 

• Opportunity for an impartial “outsider” to study the 
system 

• Feedback to: operational world 

designers 

research world (NASA) 


Figure 3 


Why are field studies important? One reason, from a researcher’s point of view, is 
that they give us a chance to get out of the laboratory into the real world. Line 
crews are a great source of information because they are the ones actually involved. 
There is a strong feeling on the part of line crews that their experience and advice 
are not being sought. Too often, only the perspective of management pilots or 
officers in the union are solicited, but line pilots are the ones who see (and know) 
the way these airplanes operate in the real world. 

There is a lot to be said for this view. For as you know, many problems do not 
appear until after the airplane has “matured.” Line flying is the acid test of design. 
You never really know how a design will work until it gets out on the line and is 
flown under a variety of conditions. Finally, one focus is to provide feedback from 
the operational world to those who are not in the operational world. 
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A final important factor is that this research is impartial. I do not design, sell, or 
operate aircraft. My purpose here is to be able to feed back the results of my 
research to those who can use it — the operational world, designers of the aircraft 
and their systems, and the research world. 


BASIC INFORMATION SOURCES 
IN FIELD STUDIES 

• Questionnaires 

• Interviews — crews 

• Interviews — check airmen 

• Attendance at ground schools 

• Jumpseat observations 


Figure 4 


A basic source of information in field studies is questionnaires. We have an 
elaborate set of questionnaires for use with volunteers. We also use face-to-face 
interviews, one-on-one, or one-on-two with the crews. I interview check airmen, 
management pilots, simulator instructors and ground crew instructors to get their 
views — what they like and don’t like. I also attended ground schools at both airlines 
and have made many jumpseat observations. 

Let me also mention the source of our volunteers. Once the study has received 
approval from management and ALPA, we appeal for volunteers with a form for 
them to sign up on. They fill in their own ID code and no copy of the ID codes are 
retained after the data have been analyzed, so we really do not know who the people 
are. We got 201 volunteers out of this process. 

Figure 5 shows all of the seats previously held on the airline. It averages about 3 
seats per person. The 727s predominate. The 727 is the only airplane common to 
the two participating airlines. It is the main source of pilots to the 757 largely due to 
seniority reasons. Although the pay scale is not much more than a 727, it’s the next 
logical step in career progression. An interesting fact is that people do not tend to 
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stay on the 757 very long. They leave the 757 to fly the heavier aircraft. When I 
went through transition ground school with 4 others, only one of them did not have 
a bid for a heavier airplane in a very short time. This, of course, creates cost 
problems in training for the airlines. 



SEATS 
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HELD 


AIRCRAFT 

CAPTAIN 

F/O 

S/O 

TOTAL 
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— 
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727 

81 
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4 

7 

13 

24 

DC-10 

7 

42 

35 

84 

747 

4 

40 

42 

86 
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0 

26 

11 

37 

TOTAL 

228 

256 

225 
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Figure 5 


Figure 6 shows the seat these pilots held immediately before going to 757 school. 
The 727 predominates. You may wonder about those people who held captain’s 
positions in heavy aircraft. There were two reasons for their 757 bids. One 
significant reason was that they got tired of international and other long flights. 
However, the major reason was that they were interested in the new technology. 
And crews were bidding downward into lower paying seats because they wanted to 
experience the 757 technology. 

The same motivation seems to be true of others. There is not much of a money 
advantage in moving from the 727 to the 757. The driving motivation was flying 
the most modem plane. It was an entirely personal motivation which very much 
affected the attitude these pilots had as they went through training and onto the line. 

Now I’d like to show you one of 36 attitude questionnaires. These are Likert or 
“intensity” scales which measure attitude. The questionnaire makes a positive or 
negative statement. The respondent either agrees or disagrees with the statement, at 


41 


some level of intensity. The scale goes from strongly agrees to strongly disagrees. 
Figure 7 is one example. 
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Figure 6 


Figure 7 shows an attitude question on the problem of error and automation. It is 
more typical of the kinds of responses in this study. The responses are almost split 
down the middle as to agreement.and disagreement There was very little strong 
agreement or disagreement on this question. You could probably not collect data 
with a more symmetrical curve if you tried. 

We should stop talking about “pilot opinion” as if it’s uniform, because the answers 
we received are varied. For example, one question involved altitude alerts. The 
MD-80 does not have an aural tone on the altitude alert and the air carrier’s fleet 
had about 136 DC-9s with aural alerts. They also had 7 DC-9-80’s (MD-80s) 
without them. When I asked crew what do you think about not hearing the aural 
warning, the answers split right down the middle. Half of the pilots said they feared 
they might now bust altitudes because they had been flying DC-9s for years with the 
aural alerts. The other half said it was good riddance. They said we’ve got enough 
tones in the cockpit already. It may not be possible to standardize based on pilot 
opinions. 
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The first series of questionnaires was in 1986 (light columns). The dark columns 
show results from series of questionnaires given in 1987. The questions were the 
same on both. 


13. We make fewer errors in the B-757, 
than we did in the older models. 


Phase 1 
n = 1 66 



Strongly Agree Neutral Disagree Strongly 
Agree Disagree 

PILOT'S RESPONSE 


Figure 7 


The advance of automation has been based largely on three assumptions which Ren 
Curry and I have questioned. The assumptions are: 

1) Automation will reduce workload. (This was the basis of the 
President’s Task Force endorsement of the 2-man cockpit; so that 
automation could take the place of the third crew member). 

2) Error will be reduced. This is a traditional engineer’s approach to 
reducing error — to automate and try to remove the human, “the 
source of the error,” from the operating loop. I question this 
because even though automation does change the nature and 
location of human error, it does not eliminate it altogether. In fact, 
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we believe that automation can be an amplifier. It tends to tune out 
small errors and create an opportunity for gross errors. 

3) Automation will be uncritically accepted by crews. We can show 
that automation is accepted by some and rejected by others, and that 
some parts are accepted while others are rejected. 

We found that there was also concern about safety. Basically, pilots liked the 
airplane and the automation. Many did not feel it was an advance in safety because 
of the opportunity for error. For example, one question was to relate an error that 
the pilot had committed or seen someone else make. A very large percentage of the 
answers were altitude bust errors which occurred for a variety of reasons including 
human error. 

An interesting thing about the 757/767 and the A3 10 automation, is that it creates an 
opportunity for new or unseen errors. For example, names of waypoints and 
identifiers must now be stored in the data base. Now pilots must be able to spell. If 
they are cleared to an intersection which is not on the LEGS page, they must be able 
to spell. It’s not unusual to hear on the radio, “ You are cleared to so and so 
intersection.” The pilot responds, “How do you spell that?” Of course, the FAA 
created this problem when they went to 5 -letter words for intersections. For 
example, if you were cleared to the Bridge intersection in San Francisco, how do 
you spell bridge? It’s BRUJ. But, in the Seattle area, you might be cleared to an 
intersection pronounced “Laker,” but spelled LACRE. 

Another example involves waypoints. A flight from Dallas to San Francisco, for 
example, passes over 2 Las Vegas’s — LVS in New Mexico and LAS in Nevada. If 
you were cleared direct to Las Vegas and picked the wrong one, there is a very 
good opportunity to penetrate a MOA (Military Operations Area). 

I rode in a 767 from Dallas to San Francisco. I didn’t say anything about the Las 
Vegas problem but I had been thinking about it. During the trip the captain said let 
me tell you something my co-pilot did last week. “We were cleared to Farmington, 
which is FMN and in northern New Mexico. My copilot reached down and punched 
in FAM and we were ready to go to Missouri. Of course the moving map display 
saved us.” The point is that a new source of potential error has been created. (Note: 
late in 1988 a pilot submitted an ASRS report. He had done just that. He punched in 
FAM and soon after noticed a distance to die next waypoint of 1 100.) 

The ALPA mapping committee is concerned with the naming issue and they gave 
me a list of interesting intersection names (Figure 8). 
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INTERSECTION NAMES IN NEW JERSEY AREA: 


BIGGE, BOGGE, BUFFY, 
HARTY, GOOFY , KITTE, 
KIP PI, FLY PI, CATTE 


Figure 8 


The examples in Figure 8 are all within about a 4 inch square on a high-altitude 
chart of the New Jersey, Eastern Pennsylvania area. Try reading that list 3 times 
quickly without stumbling over it. This, of course, is not peculiar to automation but 
it is a problem with ATC with any type aircraft. I showed this slide at a meeting in 
Washington in December in 1981. A staff member of the Canadian Safety Board 
came up later and told me that they recently had a loss of separation — a very near 
midair collision. The intersection names were confusing (BADDE, BANNY) and 
contributed to the problem. Sometimes I wonder if there is any human engineering 
at all, considering the Korean 007 incident, and with names of the tracks in the 
North Pacific like NDPPI, NINNO, NOKKA, etc. 

Figure 9 indicates the results of a question regarding workload. The distribution of 
answers is my favorite — a very nearly perfect symmetrical distribution. It shows 
the answers from qualified 757 pilots. 

Comment from audience: Doesn’t the symmetrical distribution of the answers say 
something about the question? How do you validate the question? Or does it also say 
that the question, itself, may not be important? 

Response: If you mean the wording of the question, it is all-important. 

As to whether the question, itself, is important, it is necessary to see all 36 questions 
together and then look at the ones that are grouped. This morning I’ve only been 
showing fragments of the data. 
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18. Automation does not reduce 
total workload, since there 
is more to monitor now. 
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Strongly Agree Neutral Disagree Strongly 
Agree Disagree 


PILOT'S RESPONSE 


Figure 9 


Comment from audience: Earl, one thing that this might lead us to look at is the 
amount of negative carry over from what’s called Phase I ground school/Phase II 
simulator type training where there is a very strong emphasis on using the 
automated features. Early in the introduction of our airplanes in 1982 there was a 
reluctance to accept automation, but, as the pilots became more comfortable, they 
not only accepted automation but became more dependent upon it. Some training 
programs in the initial phase of ground school had this emphasis. This question may 
indicate a negative carry over as a result of the fact that the pilot has not been 
trained to look out the window or because he just does not have the time because of 
his workload. The question is, is it pilot-induced workload or is it man-made 
workload because of the design of the aircraft? Or is the basic problem the design of 
the training program? 

The most consistent complaint we hear about the training program is that too much 
time is spent on the CDU and on the automatics, and not enough time on the basic 
flying of the airplane. 
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Response : There is considerable concern about this question of the amount of time 
spent focused inside the cockpit by both pilots when the aircraft is below 10,000 feet 
coming into a terminal area. This is something we really must explore. 

There is a movement that Ren Curry first identified when he talked about “turn it 
off’ training. As people gained experience with automation, more and more they 
were doing the opposite of what was expected. They started turning it off below 
10,000 feet coming into the terminal areas particularly in an area like Los Angeles 
where “musical runways” or changes in runway are frequent. This is a training 
problem, a standardization problem, and a crew coordination problem as well. 

Much of “who does what” breaks down in the terminal areas and if you ask line 
pilots or check airmen, the one thing they are concerned with is heads down in the 
terminal area. It is interesting that I heard the same thing 5 to 7 years ago when I did 
the MD-80 study. The MD-80 doesn’t have half of the heads down opportunities 
that the Boeing 757 does. Essentially, the MD-80 has a mode control panel, but not a 
CDU. 

That is one of the problems with the workload question. More and more people are 
saying essentially, “turn it off’ if you get busy. You can see why I call that the 
paradox of automation. (Figure 10) When the workload gets heavy you turn off the 
thing that was designed to reduce the workload. Now again, is it a training 
problem? Is it a crew coordination problem? I think more than anything else it is a 
concept problem. There is something wrong with the basic concept of the design of 
the automatic features today. And that’s why I say that years from now we 11 look 
back and call this the era of clumsy automation. 


THE PARADOX OF AUTOMATION 

“A LOT OF TIMES WE JUST CLICK IT OFF 
AND GO BACK TO MANUAL IF THE LOAD 
BECOMES HEAVY.’* 


- 757 pilot 


Figure 10 

Question: Is the automation the problem or the interface with the automation? 
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Response: I’m not making a distinction. Clumsy automation in my mind refers 
largely to the entire program. 

Comment from audience: I am confused. Are you only addressing the reduction of 
workload when you mention the possibility of using automation to get rid of 
excessive workload? If you consider the new systems automation in the cockpit such 
as the FMS and the advanced autopilot, which have not been mentioned, it is another 
story. The present transports only address the way that the flight engineer has been 
removed by management of flight engineer duties. Now we are addressing a whole 
new set of tools. 

Comment from audience: I’d like to comment in this area too. I think that the 
statement about automation — turning off the CDU’or the autopilot and using 
automatic flight coupled with a CDU was ambiguous or misleading. This is quite a 
different issue from turning off the EICAS or the other automated systems and 
flying the airplane in a truly manual state. 

Response: I don’t think anybody has any argument about the EICAS. The problem 
is control in navigation or the speed control aspects. But this is essentially an 
interface problem. The CDUs are difficult to operate and a sizeable number of 
experienced crews were saying, “when the workload gets heavy I click it off.” 

Comment from audience: Is this almost a mis-use of the automation? For example, 
the crew should not try to reprogram under those conditions. The ASRS has been 
getting a lot of incidents where the pilots say “we shouldn’t have been 
reprogramming.” In many cases, it’s not the way they were taught, but they 
reprogram anyhow. This concern about aircraft and their automation may have 
been overstated. Some of the problems we see may not be related to the automation 
itself, but arise because we use less than optimum procedures and then do not train 
very well for the less than optimum procedures. The automation is then judged as 
no good. That, of course, is very much overstated. In ASRS structured callbacks to 
pilots, we ask how they handle this very real problem. More and more pilots are 
responding, “I only reprogram when I can.” 

Response: This is a delicate balance we must reach. Another aspect is an ATC 
system that simply is not cordial to these aircraft. The classic case is coming into 
Los Angeles from the east where we are having problems, because of all the 
reprogramming required. The crew is set up for 24R and about the time you get to 
Twenty-Nine Palms or Hector, ATC switches to 25R. If the crew is a good guesser, 
they know where they will end up and are ready for about 4 routes — back and 
forth, over CIVET, coming in over Los Angeles, one more runway change. And 
that’s when there is a division of opinion. Should you click if off, or should you 
program it? There are very strong opinions about that. I don’t think we’re going to 
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resolve this question, i.e., the incordiality of the ATC system, before the end of the 
century. 

Comment from audience: Your study includes a number of pilots their 
preferences and opinions, but Figure 10 represents only one viewpoint which 
happens to be a very powerful opinion. It is probably not fair to overgeneralize, 
particularly when another pilot stated, “I can change it in 3 seconds.” Figure 10 is a 
single viewpoint whereas statistics require hard numbers. A single statement may 
give the erroneous impression that everybody is clicking it off and this could be 
incredibly damning to the system. 

Response: I agree; that’s a valid criticism. I did not mean to say that this opinion 
represented the whole population. I am trying to simply put forth a few ideas from 
the pilots I flew with and interviewed, and I’ll try to give both sides of the issue. 

Comment from audience : We understand what the pilots are trying to say in Figure 
10, but a statement like this requires more detailed information to understand what 
is behind it. Examining only one aspect of automation from one particular reply can 
be misleading. We have learned a lot regarding the interface of this particular 
system and it can be improved. We’re all working on it, but the quote on “the 
paradox of automation” must be used in the proper context. 

Response: I agree, but I want to describe the paradox of automation. Very 
frequently, but not all the time, pilots turn off the automation in response to heavy 
workload . 

Comment from audience: It is very important to be specific regarding what aspect 
of automation. Exactly which systems are they turning off? They don t turn off all 
the automation. For example, the EICAS system is still used. 

Response: Yes, that is a good point. Not all automation is turned off. Crews may 
turn off the LNAV, VNAV, and go to basic autopilot and flight director mode. 
They don’t turn off the yaw damper, for example. But there is still a paradox which 
is not a lack of training, a lack of utility or even a lack of devotion to automation. 
Some aspects of this technology are occasionally inappropriate to use during the 
required maneuvers of one particular system. 

Comment from audience: From the standpoint of our operations, until the advent of 
the 757/767 aircraft, when did anyone ever think you could make an NDB (non- 
directional beacon) approach on an autopilot? We suddenly had an airplane that not 
only gave you an autoflight system that you could do an NDB approach on the 
autopilot, but it gave you a moving map so that you not longer had to put your 
finger on the chart and follow along as you were also flying the approach. 
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We put all this information in front of the pilot and showed him how to use it. He 
was trained on this tool and given the “Madison Avenue” approach about how great 
automation was by the manufacturer, the vendor, the public in general. Then, when 
the pilot flew into the Miami area, the controller told him, “maintain 220 knots until 
further advised and expect clearance for an NDB approach.” And shortly another 
ATC clearance to “Slow to 170 knots.” At that point, the pilot must slow to 170 and 
configure the airplane. But, the autopilot won’t respond fast enough. The only way 
the pilot can get out of this dilemma is to turn the automatics off, pull the speed 
brake, drop the gear, and get caught up. 

It’s not a failure of automation. It is because the man and the machine have not been 
able to interface with the ATC system. It’s not a case of whether you have an EICAS 
or whether you have a yaw damper system. It’s a case where the human, machine 
and the operational environment are not compatible. That’s the real question. So, is 
that a paradox of automation? Quite certainly it is. 

We train crews, give them tools, show them the disciplines and procedures, but they 
cannot make effective use of these techniques in the field. At die end of a 15 hour 
flight, would you rather do an NDB approach on the autopilot or would you rather 
do it by hand? But to force the choice of the hand flight mode, because the situation 
is not working the same as the training courses, creates the problem. Therein lies 
the paradox. 

Chair’s comment: The idea of Earl’s presentation is to raise some of these issues and 
by the discussion, I believe we have identified an important issue. I suggest that we 
summarize this issue and continue with Earl’s presentation. 

Comment from audience: I’d like to support the previous statement. There is a 
paradox. The problem was correctly stated as well as the solution. It is working 
with ATC. An approach into Los Angeles is a prime example. 

Comment from audience: One brief comment, just for balance. Earl mentioned that 
pilots haven’t seen controllers in the jump seat lately. I’m getting exactly the same 
response from the controllers in the Centers. It has been years since the pilots have 
been in the Center; it goes both ways, which points out the need to look at the system 
in terms of the mis-match. Things like simply asking controllers to do more is not 
the answer to the problem. 

We can ask both people to do more. More controller jump seating and more pilot 
trips to the Center. will certainly help coordination. 
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Comment from audience: I’d like to make an observation. Several times it has been 
mentioned that one of the problems is an ATC system which is not friendly to 
automated airplanes. This may suggest a conclusion that it is OK for the older 
airplanes. But the last minute runway shift is as bad for the 727 as it is for the 767, 
because the crew has still got to pull out the chart and do the briefing. We should not 
conclude that ATC should be more sensitive to the automated airplane or that such 
aircraft should be given special treatment. 

Response: This is not a single problem. The fundamental problem is the ability to 
deal with everything from Lear jets to 767s, right on through the system and I don’t 
see a change coming in the near future. 

There is another problem area regarding what people have to do to “cheat” on the 
automated system. My students know how to “cheat” the computer, but I didn’t 
think I was going to see it in the 757s. Things like getting down early so you can 
make it work. For example, crews do ingenious things like programming fictitious 
tailwinds, or inserting an altitude for thermal anti-ice (TAI) (but not turning it on) 
so it will get the aircraft down earlier. 

I would like to discuss training next. This is an area that requires a lot more 
thought. There is a change in training for automation that may be qualitatively 
different. Questions such as, “would you introduce the automation first as most of 
the airlines do now?.” This is a basic question, but we really don’t know the answer 
at this time. 

Skill deterioration or more dramatically, automation apathy, is another issue. In the 
field studies we have done, I don’t see any such signs. Pilots are not concerned about 
it as long as they personally keep up their skills by hand-flying. In the MD-80, the 
pilots expressed concern, but when they went back for their first Proficiency Check 
(PC) (in a DC-9-30 simulator), they had not suffered any skill loss. In fact, they said 
they flew better when they went back to the -30 simulator and did their first PC than 
before they had -80 time. 

Question : Were they flying both airplanes? 

Response: Yes, they were at that time. Later, about half way through the study, they 
flew in separate schedules, based on flying only the -80 for 9 months. Each of the 
pilots works out his own “training program” using hand-flying, flight-director 
only, or occasionally raw data only. I called it the “Personal FAR” in the -80 study. 
About 90 percent of the people do this. Another related issue is a copilot who flies 
automated aircraft and then suddenly qualifies as captain on the DC-9- 10 or 737- 
100. Skill loss has not been the problem, but our methodology may not be very 
sensitive to it. It’s always a pleasure to find something that is not a problem. 
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Question from audience: Have you looked at carriers that prepare their pilots to fly 
automated all the time? 

Response: The carriers differ on this. As a matter of company policy, one airline 
requires that crews do not use it all the time. Another company, for example in the - 
80, had a “we bought it, you use it” policy. However, the pilots would simply click it 
off and fly manually, when necessary. 

With respect to training, the opinion of one flight manager was that there was a too 
rapid introduction of the technology into the training program, and not enough 
information on the basic airplane. 

Another interesting comment from a captain about PCs in checkrides. He said, 
“Formerly, when I went for a checkride, the FAA was always turning things off. 
Now they tell me I have to turn things on. They don’t want to see you operating an 
airplane without automation.” 

This comment in Figure 1 1 came from a young first officer and in my opinion, it 
generally reflects pilot attitude. They all thought that they were “going back” — 
going back to a 727, “going back” to a DC- 10. It was a phrase we heard often. 


“I’LL TELL YOU WHAT IT’S LIKE TO GO FROM 
THE 757 TO THE 747: IT’S THE GREAT LEAP 
BACKWARD” 


Figure 11 

As a final note, Figure 12 summarizes the high enthusiasm for the 757, although I 
have mentioned some reservations about the safety aspect. Workload may be 
increased or decreased. Of course, it appears to be increased at the phase of flight 
where you would not want an increase, and a decrease at the phase of flight where 
you would not want a decrease. I don’t think that is a reflection on automation, 
rather that it is in the nature of the flight itself. It’s a major challenge for those 
designing the future systems. 
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Some first officers are very fast. They can have route 1 and route 2 both loaded 
before I can even see what they are doing. 


There is a great variety in time required. There can be differences between captains 
and first officers and the speed of each group. One person can be a whiz on 
computers and really loves it while another person may have a different kind of 
wisdom. You can see it in the training. And, of course, I only observed a small 
number. I talked to the simulator instructors and training captains and they 
confirmed this. As you watch people in the initial phases of ground school, the first 
officer learns quickly, but some of the captains had difficulty. But in the two weeks 
after they left ground school and went to the simulators, the captains had 
accelerated — taking advantage of their experience. In the simulators, the captain 
suddenly caught up. 

Question: Do you think the term programming, or reprogramming, is the correct 
word to use in the cockpit? I think we should not use those two words. 

Response: Why is that? Because you don’t consider it “programming,” but rather 
“loading data” or something like that? 

Comment: It’s an airplane, not a computer. I think those two words are dangerous, 
if one gets used to them, because we lose track of the fact that we are still flying an 
airplane. 

Comment: In our early training, the primary reluctance for the captains was over 
this issue. The word “program” placed a cloud of uncertainty, especially over the 
older crew members. In 1982, I went through training with a group of people 
whose average age was 48-53. They were very, very reluctant. Some of these 
people had not been through ground school in 16 or 18 years. They approached the 
training in a very timid manner, and when the word “programming” was used, they 
either literally had to be held back from pressing the execute button too soon or 
would have paralysis when it came to placing their finger on a key and making a 
keystroke on the CDU. They could not bring themselves to do that because of, 
again, the specter of “programming.” 

Comment: If there could be a better word or term applied to it, I believe we could 
reduce probably 40 percent of our problems. I tell our people, “slow down 10 
percent and improve your accuracy 40 percent,” and you could probably take away 
another large percentage when you simply change the terminology. 

Response: There are still difficult problems to solve on the CDU, like entering a 
route with two jet airways intersecting each other. And, you may not want to call it 
programming, but that’s what it is. It is not simply data entry. For example, there 
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If the money and quality of trips 
were the same, what would be your 
first choice of aircraft to fly? 



AIRCRAFT 


Figure 12 

I have data that I don’t have time to present to you today, but it will be published in 
the final report for the research project. The final questionnaire will have 
information on how many ADF approaches, VOR, localizer, CAT-II, CAT-III, 
autoland and “man-made” approaches. 

Question: On the programming of the CDU, from your observation, what is the 
distribution on the time required to reprogram the CDU? For example, in a musical 
runway situation requiring reprogramming. 

Comment : You know, of course, that this is not the only time that reprogramming 
is required. Reprogramming time is largely a function of experience. There seems 
to be a critical point at about an experience level of 200 hours. It was about 50 hours 
in the MD-80, and the differences were not as great as with the 757. But in the 757 it 
took about 200 hours to get really proficient. Some of the pilots are very proficient. 
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are two jet airways and you have to fly one, and then change to the other, and there 
is no named intersection. In my mind, that’s programming. 

Susan Norman: In the new technology, we should be clear about the meaning of 
terms. We have prepared a list of terms (see appendix) and hope that they will be 
useful. I hope you will consider Earl a valuable resource for discussion, but in the 
interest of time, I must conclude this discussion. 
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THE ADVANCED AUTOMATION SYSTEM (AAS) 
FOR AIR TRAFFIC CONTROL 

Alden Lerner, Federal Aviation Administration 
Washington, DC 


PURPOSE 

The advanced automation is tomorrow’s air traffic control system. It will provide a 
new automation system that includes new computer hardware, software and 
improved controller work stations. The advanced automation system will provide. 

• the capacity to handle the projected traffic load through the 1990s 
and beyond, 

• the capability to perform new functions to be introduced into the 
ATC system through the 1990s, 

• increased productivity through the introduction of new sector 
suites, 

• a high degree of reliability and availability, and 

• the capability for enhancement to perform other functions subse- 
quently introduced into the system. 

APPROACH 

The advanced automation system (AAS) will be designed through a top down, 
evolutionary, total system approach. Controller sector suites will consist of 
common consoles used for both enroute and terminal (approach control) functions. 
They will incorporate an improved man-machine interface, including the use of 
color displays and electronic presentation of flight data to enhance controller 
productivity. The advanced automation system will make possible the full 
integration of enroute and terminal operations in the area control facilities. 

Transition to the AAS will consist of five phases: 

1) implementation of the Peripheral Adapter Module Replacement 
Item (PAMRI), 
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2) implementation of the initial sector suite system (ISSS) to provide 
new controller work stations, 

3) implementation of terminal advanced automation (TAA) functions 
using AAS hardware and software, 

4) implementation of tower control computer complexes (TCCC), and 

5) implementation of Area Control Computer Complex (ACCC) for 
the remaining AAS enroute functions. 


Phase one, implementation of the Peripheral Adapter Module Replacement Item 
(PAMRI), which includes replacement of the PAM and Data Receiver Group 
(DRG) equipment, will be implemented prior to ISSS equipment delivery. This will 
provide an enhanced ability to interface with additional radars, and win provide a 
capability for use of higher data transmission rates for radar site interfaces. 

In the second phase, initial sector suites will be installed in enroute facilities served 
by the host computers. The sector suites (work stations) will be the first AAS 
components to be operational, providing early gains in controller productivity. The 
consoles will feature large, multiple color displays that will provide situation 
traffic, weather, and flight data as well as a “look ahead” planning capability. 
Although each console will have its own embedded microprocessors to drive the 
displays and perform related tasks, most of the data processing required for 
controlling traffic will be performed by the host computers. First delivery of an 
ISSS will be to the FAA Technical Center in Atlantic City in September, 1990, 
where it will undergo extensive test and evaluation. First site delivery to the Seattle 
Center is scheduled for April, 1 992. The equipment is expected to be operational at 
all sites by June, 1995. 

The third phase will be the implementation of the TAA for TRACON functions. 
Following the transition to ISSS for enroute control, the old control rooms will be 
refurbished to accommodate the additional sector suites necessary to provide 
approach and departure services at the approximately 200 airports that now have 
their own terminal radar control rooms. Deliveries of new AAS hardware 
processors, software, and additional sector suites to support terminal operations 
will begin in 1994. 

The fourth phase will be the installation of TCCCs in selected airport traffic control 
towers. The TCCC will include new controller position consoles with supporting 
computer hardware and software and new electronic displays for flight data and 
radar information. Controllers will use the radar data as an aid in tracking aircraft 
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in the terminal area, as they may be required to provide limited approach and 
departure services. The contractor will begin delivering TCCCs to upgrade the first 
150 airport towers included in the basic contract in 1994. The last site (258th, if all 
contract options are exercised) would become operational in 1999. 

The fifth phase in the evolution to a full AAS will begin in 1994 with the installation 
of the remaining computer software required for the enroute air traffic control 
functions (ACF). Additional sector suites will be installed to enable conversion of 
today’s ARTCCs into tomorrow’s ACFs. The hardware/software associated with 
this step will be named the area control computer complex (ACCC). Once this is 
completed, the host computers at each location can be removed along with the 
current back-up system in the enroute centers. The completion date for this phase is 
August, 1998. 

Included in this phase would be the addition of the Automated Enroute Air Traffic 
Control (AERA) software into the system to aid controllers in effective route 
planning. Known as AERA-1, it will probe requested flight routes to detect 
potential conflicts with other aircraft, violations of protected airspace, and 
conformity with air traffic flow restrictions. With this information, the controllers 
can select the safest and most fuel-efficient route available. 

Future AAS enhancements not covered by the contract will include AERA-2 and, 
possibly AERA-3. AERA-2 would present controllers with solutions to the 
conflicts detected by AERA-1 and, then, pass along their decisions to pilots by 
digital data link. AERA-3, which is still in the concept stage, would include some 
degree of autonomy for the AAS computers to detect and resolve problems, make 
decisions and provide clearances to pilots under human direction, but without 
human intervention. 


PRODUCTS 

The scope of the AAS project includes: 

• Advanced automation system design 

• Advanced automation system software for both terminal 
and enroute ATC operations 

• Advanced automation system computer hardware 

• ISSS (20 Continental U.S. ARTCC) 
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• ACCC (ACFs — including Anchorage, Honolulu, and the 
New York TRACON) 

• Support systems at the FAA Technical Center and the FAA 
Aeronautical Center. 


IMPACT 

The introduction of the new controller work stations and new communications 
networks will greatly impact the controller’s physical environment. Further, the 
organization of the new equipment into ACFs which will result in the co-location of 
terminal and enroute functions in the ARTCC will impact the procedures for 
managing the duties of air traffic control, maintaining the systems, and organizing 
the work force. 


The Advanced Automation System is tomorrow’s air traffic control system. Its 
sophisticated equipment and programs will improve upon present air traffic control 
by: 

• enhancing flight safety with new automatic separation-assurance 
techniques, 

• increasing flight efficiency through more direct, conflict-free 
routing, 

• reducing congestion and delays through better traffic-metering 
techniques, 

• increasing controller productivity through new controller work 
station, 

• handling projected air traffic growth without corresponding in- 
creases in personnel, 

• providing a system life of 20-30 years; new hardware/software can 
be added to the basic design, 

• tying together all of the FAA’s primary enroute and terminal air 
traffic control facilities into an integrated, automated system, 

• permitting the consolidation of all radar services into approxi- 
mately 23 strategically located Area Control Facilities (ACF), 
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• providing greater system reliability through a requirement that the 
AAS be available 99.9995 percent of the time (maximum down 
time of about 2 1/2 minutes per year). 

AAS DELIVERY SCHEDULE 

Initial Sector Suite System 

Delivery to FA A Technical Center - September, 1990 
Delivery to first ARTCC - April, 1992 
First site operational - October, 1993 
Last site operational - June 1995 

Tower Control Computer Complex 

Delivery to FA A Technical Center - April, 1992 
Delivery to first airport tower - March, 1994 
First site operational - February, 1995 
Last site (150th) operational - March, 1999 

Terminal Advanced Automation System 

Delivery to FA A Technical Center - December, 1992 
Delivery to first ARTCC - march, 1994 
First site operational - February, 1995 
Last site operational - February, 1998 

Area Control Computer Complex 

Delivery to FAA Technical Center - January, 1993 
Delivery to first ARTCC - September, 1994 
First site operational - March, 1996 
Last site operational - February, 1998 
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Invited Paper 


THE EFFECTS OF AUTOMATION ON THE HUMAN’S ROLE: 
EXPERIENCE FROM NON-AVIATION INDUSTRIES 


Dr. David Woods, Ohio State University 


Susan Norman: Automation technology has been employed by other industries 
for many years. Although the operating environments are not exactly like aviation, 
the experiences regarding the technology itself are surprisingly similar. Dr. Woods 
was asked to select a few representative examples with relevance to aeronautics and 
to draw some general conclusions about their applicability in the transport aviation 
environment. 


Dr. Woods: 

I have been asked to comment on the effects of introducing new automated 
technology. Rather than start with broad generalizations, I have selected a variety 
of specific, actual cases where field studies or controlled studies have been 
conducted to determine the effects of new technology on both productivity and the 
quality of human performance. The cases that I have chosen to discuss are relevant 
in some fashion to aviation, and they are listed in Figure 1. 

An important point is that these examples are based on field study results, not 
opinions. Of course, there can also be questions of interpretations in field studies. I 
was perso nall y involved in some fashion in several of these studies. Some are based 
on reviews of other people’s studies on the effects of automation. It will only be 
possible to discuss each case briefly but more detailed reports are included in the 
references. 

The term “automation” has been used so much that it has taken on several meanings. 
In several of the examples which follow (process control and computerized 
numerical control or CNC), automation refers to autonomous machine systems 
because the jobs are performed in a semi-autonomous way, without direct manual 
intervention. The other examples concern “automation in the sense of new 
information systems capabilities such as diagnostic systems. 
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Studies or Field Experience on the 
Human Role in Highly Automated Systems 

• Computerized Numerical Control in manufacturing 

• Banking information systems and processes 

• Steel processes 

• Nuclear Industry: 

• Computerized alarms systems 

• Disturbance Analysis systems 

• Computerized procedures 

• Expert systems 


Figure 1 


TECHNOLOGY CENTERED AUTOMATION 

These cases all illustrate a common underlying philosophy of automation which has 
been called “technology centered” automation. The assumptions of this approach to 
developing new levels of automation are noted in Figure 2. 

The issue is not to automate or not to automate; it is how to automate. How should 
the level of technology be increased? What should we do with the technological 
capabilities? Choices can be made about how to introduce the technology which 
require careful thought. There is no technological imperative — only one way to use 
technology. 

The fundamental assumption behind most automation development is that people 
and machines are comparable; one can be substituted for the other on each subtask; 
and the tasks to be performed are independent. In other words, changing the 
allocation on one task has no effect on other tasks. It has been difficult to make 
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progress in the field of human factors because we have been locked into this “can we 
substitute a machine for a person’’ approach to function allocation. 


Technology Centered Automation 
philosophy of person-machine comparability 

allocate to people tasks that are expensive or difficult to assign to 
machine 

system problems are solved by attempting to shift human functions to 
the machine 

person is often a convenient and cheap manipulator or perceiver 

de facto human role: “plug the holes in the thoroughness of the 
designer’s work” 


Figure 2 

The result has been that tasks which are expensive or difficult to assign to the ma- 
chine are allocated to the people. We tend to automate the tasks that are automatable 
at a reasonable cost. If there are system problems, they are usually dealt with by 
transferring more human functions to the machine — solve performance problems 
by reducing the human’s role in the system. There has been no global systems 
context where the person and machine work together. 

There is also a tendency to use the person as a convenient and cheap manipulator or 
perceiver for the machine, because it is difficult to automate perception. This is 
particularly true in the development of expert systems as we shall see later. 

In the technology centered approach to automation no thought is given to the 
human’s role in proper system function. Instead, as Jens Rasmussen has pointed out, 
the de facto human role is to “plug the holes in the thoroughness of the designer s 
work” (Rasmussen, 1979). 
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COMPUTERIZED NUMERICAL CONTROL 


One of the early automation cases, which is often misdescribed in terms of what 
actually happened, involved computerized numerical control (CNC). This case, 
summarized in Figure 3, has been one of the most investigated cases of changes in 
automation. The original goal was simply to eliminate the skilled machinist to save 
money. What actually happened depends on the exact machine application, but in 
general, the human role, as well as the skills needed to carry it out, changed (cf.. 
Noble, 1984; Corbett, 1985; Kidd, 1988). The operator was now responsible for 
preventing gross machining errors such as machining the wrong part. But if the 
operator was not skilled and knowledgeable in the tooling process, unscheduled 
down time and gross machining errors occurred. 


Computerized Numerical Control 
original ggglsi eliminate skilled machinist 

asinal results: changed human role and skills to avoid unscheduled 
downtime and to minimize the costs of machining errors 

critical human role is to adapt to unanticipated variability with success 
depending on: 

• machinist learning and inferring what is the 
computer plan , 

• where it is vulnerable to breakdown in the face 
of different circumstances (e.g., tool wear), 

• how the computer plan is progressing in a par- 
ticular case, 

• devising and using ways to directly or indirectly 
control the CNC system 


Figure 3 

The critical human role was to adapt to “unanticipated variability.” Conditions still 
arose in the machining tool process that went beyond the capabilities of the CNC 
machine programs. The human had to help the machine to adapt. The machinist 
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now had to understand something about the plan resident in the computer program. 
He did not have to be able to duplicate it or to write the program, but he did have to 
understand what it was trying to do — its intentions. He had to understand where it 
was vulnerable to breakdown and what factors gave it trouble. 

Tool wear, a classic case particularly in the earlier systems, was a factor that could 
challenge the automated systems. The operator monitoring requirements went up — 
as the machining progressed through a run, the operator needed to understand how 
the plan execution was progressing. Was it going off track? Was it starting to have 
problems? This required supervisory control (a theme which repeats in many of the 
cases to follow). As a supervisor, the operator now had to adjust and redirect the 
system occasionally. The designers, however, rarely provided convenient ways to 
accomplish this because they did not think about the machinist’s role. It was up to 
the machinist to devise new ways to control the CNC systems. 

BANKING INFORMATION SYSTEMS 

Another example is a study of a shift in the information technology used in the 
banking industry (Adler, 1986; summarized in Figure 4). Again the exclusive goal 
was to reduce the number and skill levels of the people in the system. The term 
“peripheralization” of the human role was first applied by Adler during this study. 
What he meant is that, as the level of automation increases, the person’s role is to 
manage and care for the health of the automated system. The new human role was to 
manage the operational environment so that the system would stay within its range 
of capabilities. 

While there were many detailed variations in the skill consequences of this 
particular example, there was an overall increase in skill requirements, because the 
main role of the people in the system had changed. They were primarily dealing 
with anomalies around preplanned routines. There were many preprogrammed 
specific banking transactions, but customers often had situations that were a little 
different. Somehow the bank operators had to understand what each special case 
meant in terms of the automated system. They needed to understand the banking 
processes as well as the automation. They could no longer have a narrow task 
orientation as they had in the pre-automation days. A broader perspective of the 
whole system was now required. 

There was also a great deal of concern because the automated system was much 
more fragile. The degree of inter-coupling or interaction between the system parts, 
as well as the level of integration among the various aspects, was increased with 
more automation. 
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Banking Information Systems: 

Original 2 oal: reduce number and skill levels of tellers 
aclual zzsult&i 

1) peripheralization of human role 

2) increase in teller skill requirements: main role is to deal 
with anomalies and contingencies around preplanned 

transactions 

3) fragility of new system due to increase in level of inter- 
coupling 

4) data integrity is a major issue in part due to low de- 
tectability 

5) new error vulnerabilities! consequences (error fre- 
quency/cost relationship changes) 

6) need for greater training in several areas including the 
overall computer processing system and accounting 
procedures to avoid or correct errors 

7) in general, the increase in level of automation produced: 

• new types of task responsibility, 

• new degrees of abstractness of tasks , 

• new levels of task interdependence. 


Figure 4 


Data integrity also became a major issue. A tremendous amount of effort was spent 
to be sure that the data in the system were accurate. If they became corrupted, wide- 
ranging effects on the system were possible. This was important, in part, because of 
low error detectability. The result was an increase in the need for “situational 
awareness,” i. e., an understanding of the state of the overall system and what it is 
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doing because errors must be detected. These errors, in this case, are not 
necessarily human errors, but simply bad data in the system. 

Another common theme is that changes in level of automation produce a shift in the 
kind and cost of errors. It was not simply minimizing errors. For the first time, 
new kinds of errors were possible while others were made impossible. The 
vulnerabilities and consequences of errors had changed. The frequency of some 
kinds of errors may have been reduced, but some of the new failure forms had 
different, higher consequences. The concern was the fragility of the system. It was 
now difficult to localize and contain errors, since they tended to spread throughout 
the system in ways that were hard to detect. Usually mistakes were apparent only 
when something came crashing down, such as a customer related problem. 

As a last point, the need for greater training was not anticipated and there was quite 
a bit of scrambling. Again, remember that the initial reason for instituting 
automation was to save money by decreasing the skill requirements. But what 
actually happened was people had to understand more about the overall process 
accounting and the banking processes. They had to have some understanding of the 
overall computer processing system. Otherwise, they could not do their job of 
helping to avoid or correct errors in the system. 

In summary, this example illustrates how shifts in level of automation can produce 
new kinds of task responsibilities and new levels of abstractness which in turn 
require more conceptual skills on the part of the system operators. 

PROCESS CONTROL INDUSTRY 

A classic example of automation in the process control industry occurred in a steel 
plant (Hoogovens Report, 1976; summarized in Figure 5). Again, the original goal 
was to reduce the number and skill level of the operators. The actual results were 
quite different. For a period of time, down time was actually increased due to 
automatic system breakdowns. An in-depth study of the effects of automation in this 
case was prepared and the report states, “the need for the operator to intervene 
directly in the process is much reduced, but the requirements to evaluate 
information and to supervise complex systems is higher.” 
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Steel Processes 


original goal: reduce number and skill of operators 

actual results: increased downtime due to system breakdowns 

“The need for the operator to intervene directly in the process is much 
reduced, but the requirements to evaluate information and supervise 
complex systems is higher ” 

automatic control or manual control ; no provisions for supervisory 
control 

• no mechanisms for operators to under- 
stand or track what the automatics were 

doing 

• no mechanisms for operator to direct the 
new automation 

• authority /responsibility double bind. 


Figure 5 

Again the theme is the same as in the banking case. The problem was the designers 
had designed the plant to work in one of two modes: either automatic control or 
manual control. Obviously, there was a manual backup to run the system, but there 
were no provisions at all for supervisory control and no mechanisms for the op- 
erator to understand or track what the automatics were doing. In fact, initially, the 
operator received barely any training or information about what the automatics 
were about at all. In short, there were no mechanisms for the operators to direct the 
new automation. 

The situation was made even worse by an authority/responsibility double-bind. The 
operator was (or thought he was) responsible for the proper operation of the 
system. The effective authority was, however, in the hands of the machine 
automatics. The operator was there just in case anything happened or to help the 
automatics in situations that were not practical to automate at this time. In actuality, 
the operator was unable to coordinate control of the system. The resulting level of 
performance was the performance of the automatics alone. Unanticipated situations 
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arose which went beyond the capabilities of the automatics. Furthermore, when 
trouble arose, the consequences tended to be broader because the system was more 
integrated following the increase in automation. The overall result was an increase 
in down time after the introduction of the automation. 


AUTOMATED FACTORIES 

A case, closely related to the Hoogovens experience that is going on today, is in 
process manufacturing, where the objective is the automated, or “lights out,” 
factory (if there are no people, then there is no need for lights). Steve Miller at 
Camegie-Mellon has an interesting program of research in progress in this area 
(cf.. Miller & Bereiter, 1986; Bereiter & Miller, 1986; Bereiter and Miller, 1988). 
He is working with a major corporation and some of their automated lines to study 
the effect on the human’s role. Again, the inter-coupling or relationship between 
the system components is increased through new automation, and the critical human 
role becomes fault management such as avoiding unscheduled down time. However, 
there has been very little support provided for this human role, as summarized in 
Figure 6. 


Automated Factories 
original eoal: lights out factory 

results: automation increased level of intercoupling 

critical human role is fault management to avoid unscheduled 

downtime 


Figure 6 


NUCLEAR INDUSTRY: COMPUTERIZED ALARMS 

In the nuclear industry, there are a variety of cases, and I have selected a few which 
are representative of the important issues. These examples can be hard to dig out 
because, in this world as in many others, no one wants to discuss technology 
failures. The feeling is it would be better if, like old generals, they just faded away. 
No one wants to investigate what happened or why, but the lessons for the future are 
important 
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In the British nuclear power industry in the seventies, the tile annunciator alarm 
systems were computerized (Pope, 1978). A tile annunciator alarm is an an 
engraved tile that is back-lit, and each tile represents a setpoint or component status. 
If the variable crosses the setpoint (or if a component is in a particular state — pump 
off), the tile lights up. If there is another setpoint on the same variable, it is 
represented by a separate engraved tile. There can be thousands of these tiles in a 
nuclear power plant, and an avalanche of alarm signals occur during plant upsets. 

There are serious difficulties associated with fault management in this situation 
related to data overload (Lees, 1983). A computer solution was devised to deal with 
the shortcomings in the old system. The computer based alarm system contained the 
same raw alarm data — setpoint crossings and component status changes. This data 
was now organized in a chronological list (in part, because it was easy to build with 
the technology of the day). 

When the new computerized system was installed, it was discovered quickly that the 
operators could not effectively monitor and track plant state during upset 
conditions. The old pilot annunciator system had to be re-introduced. The reason 
for this failure was that the designers of the new system did not understand some of 
the serendipitous benefits of the old tile annunciator system (the inherent spatial 
organization) which were eliminated by the shift to the new technology. In the old 
tile annunciator system, even though there were huge numbers of signals, they were 
spatially distributed. Each tile was spatially assigned to a location and the operators 
could determine some things about the overall problem based on the pattern of 
lighted tiles. 

The old system made use of human pattern recognition capabilities, and, with 
experience, operators could leam to recognize patterns (Kragt & Bonten, 1983). A 
chronological list of very raw alarm information, however, comes in fast and gets 
very long, very quickly. The operator had to refer back down the list and to scroll 
back and forth through the screens trying to find out what happened and to build an 
integrated picture out of the raw alarm data. This was extremely difficult to do with 
the chronological organization, exacerbating and not relieving the data overload 
problem. 

This theme, summarized in Figure 7, is common in many of the information system 
oriented increases in automation. Designers will be technology driven and build the 
automation that makes the most sense in terms of cost and the use of the technology. 
But the cognitive consequences for the problem solving task, fault management in 
this case, are rarely considered. 
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Nuclear Industry: Computerized Alarms 

shifted from tile annunciator alarm systems to a form of computer 
based alarm system 

original goal: improve alarm systems, use computers 

QZtUOl results: the tile annunciator system had to be reintroduced 

serendipitous benefits of the tile system were eliminated in the 
computerized system that greatly increased the difficulty of fault 
management 


Figure 7 


Another example occurred on ships when a new electronic display and control 
system was introduced into the engine room. Automation designers frequently try 
to finesse the cognitive consequences of changes in the person-machine system by 
relying on the flexibility of computer based systems-the designer’s only 
responsibility is to make all of the data available and accessible; it is the domain 
practitioner’s job to find the right data at the right time. However, a case described 
by Moray (1986) illustrates how flexibility alone is not enough in the development 
of more automated information systems. 

In this case, a new, fully computerized ship engine control room was developed. 
There were three CRT screens, and the operator could call up a variety of computer 
based displays and controls on whichever CRT he or she desired. A human factors 
review of the system predicted that, at some time in the life of this system, the 
operator would call up the computer display for the starboard engine controls on 
the port CRT and the computer display for the port engine controls on the starboard 
CRT-a violation of stimulus-response compatibility guidelines. This could lead to 
an execution slip where the operator would unintentionally manipulate the wrong 
ship engine. 

Shortly thereafter, during simulated shiphandling with the new system, this 
situation arose and the predicted result followed. Alarms indicating starboard 
engine trouble occurred. The operator correctly diagnosed the situation and 
attempted to control the starboard engine. He manipulated the engine controls on 
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the starboard CRT display which display the port engine controls. If this had 
occurred at sea during a difficult navigation period, the ship could well have run 
aground. 

NUCLEAR INDUSTRY: DISTURBANCE ANALYSIS SYSTEMS 

The next example. Figure 8, is one of the best illustrations of a failed attempt at 
automation which faded away without comment or investigation. It was several 
projects that went on about the same time in three different countries to build 
“disturbance analysis” systems in the nuclear industry. The projects were attempts 
to address problems in operator diagnosis of faults and the deficiencies in alarm 
systems by automating fault diagnosis (artificial intelligence techniques were not 
used). 


Nuclear Industry: 

Disturbance Analysis Systems 

multi-year, multi-million dollar projects in U.S., Germany, Japan, 
circa 1980-1984 

Original goal: carry out automatic fault diagnosis; reduce operator role 
in diagnosis 

aolml results: 

1) unable to automate fault diagnosis and did not 
make the operator a better diagnostician 

2) exacerbated operator data overload 

3) projects transformed or abandoned 


Figure 8 

However, diagnosis in a large complex system like a nuclear power plant is 
inherently a very difficult task (Woods, 1988). For example, there are typically 
multiple faults and interacting factors that explain the pattern of symptoms. The 
disturbance analysis projects were unable to do automatic fault diagnosis and they 
did not make the operator a better diagnostician. 
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In part, this occurred because the data overload on the operators was exacerbated. 
The disturbance analysis system generated large amounts of potentially useful, 
more integrated information. But now the operator had the task of interpreting all 
this data, finding out what was useful and integrating it with the large amounts of 
time varying data available from other information sources. While this experience 
has generated more interest in human-computer techniques for managing large 
amounts of dynamic data, the lessons are largely going unnoticed. Today, another 
attempt is underway to automate diagnosis via artificial intelligence techniques. 
Interestingly, the label “disturbance analysis” is taboo in the nuclear industry. 

COMPUTERIZED PROCEDURES 

Another case (Figure 9) concerns computerizing procedural information. There 
were a variety of goals such as improving data retrieval from large libraries of 
procedural information. In the end, however, the attempt was abandoned because 
people got lost in the system. 

The data base application in question was a computerization of paper-based 
instructions for nuclear power plant emergencies. The system was based on a 
network data base “shell” with a built-in interface for navigating the network. The 
shell already took into account the basics of human-computer interaction so it was 
assumed that all that was needed was to enter the domain information. However, 
several factors combined to produce a keyhole effect. These factors were the 
organization of the network, the standard navigation features, and the fact that 
power plant incidents are dynamic and new events can occur which change how the 
operator should respond. 

In summary, the operator could only see one procedure step at a time; it was 
difficult to refer back to a previous step or to look ahead to see what steps have to be 
done next (Elm & Woods, 1985). There was no attempt to provide the operator 
with a broad picture of the response plan being executed or where it was going. 
Trials with the system revealed that the people who used the database shell to 
generate the system got lost. The people who wrote the procedures and tried to use 
this system got lost. Experienced operators, who knew what step in the procedures 
should be executed given the current plant state, also got lost. 
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Computerized Procedures 


Shift from paper based to computerized procedures. 

original gOOhl improve maintenance of procedure information, 
improve operator procedure retrieval, ensure rote procedure 
following, use new technology 

flcfiia/ resn/fs: attempt abandoned due to user getting lost in the 
network of data 


Figure 9 

The end result was that the attempt to computerize the procedures in this way was 
abandoned. A side note is that a redesign was done based on a proper cognitive task 
analysis of procedure usage. The new design utilized several techniques to avoid 
keyhole effects and to support the user. Interestingly, almost all of this design could 
have been implemented within the base capabilities of the interface shell. 

EXPERT SYSTEMS 

The next case addresses expert systems for troubleshooting. In this one I and my 
colleagues were able to investigate how technicians used an industrial expert system 
developed from a shell in the standard iterative refinement approach to 
troubleshoot an actual electro-mechanical device (Roth, Bennett & Woods, 1987). 

The original goal was to reduce the technician’s skill requirements. A second goal 
was to provide management with extremely tight control over the technicians who 
were responsible for troubleshooting the devices. 

The expert system was designed to be a stand alone problem solver who would 
simply output a solution. But, what was the person supposed to do? The person was 
expected to go to the remote site and dial into the expert system to initiate a run. The 
machine would then use the person as hands and eyes; the person’s role was simply 
to serve the machine, to collect data, to carry out various kinds of tests, to make 
observations about device behavior and, finally, to implement the repair selected by 
the machine expert. 
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If this model of problem solving was correct, the data that we collected should look 
like that in Figure 10. The machine expert would direct the technician to perform a 
series of data gathering activities (observe, measure) until the machine produced a 
hypothesis and the operator effected the directed repair (e.g., replace the bad part). 
Essentially, the expectation is a straight, linear execution of the machine’s 
instructions until the solution is found. 

How much of the time did this happen? About 20 percent of the time. Almost 80 
percent of the time, the protocols looked like the one in Figure 11. The machine 
generated multiple hypotheses, not a single one. One part of the human role was to 
filter the machine’s solutions, and decide which was really the right one. People had 
to figure out if the machine was on track; as a result, they had to go through their 
own diagnostic process. Were the machine generated hypotheses inconsistent with 
the operator’s experience? If so, the operator would need to supervise the machine. 
But, remember that the machine design provided no mechanism for the technician 
to direct the machine or to gain access to the knowledge or tools available within the 
expert system. So the operators tried to adapt and invent ways to interact with the 
system. Specifically, the operators tried to trick the automatics in order to get the 
machine to do what needed to be done. In one case, when the machine response did 
not make any sense, the operator went back a step and tried the opposite answer to 
see what would happen and if the machine’s behavior would be more plausible. 

The actual results were that a successful diagnosis required significant technician 
participation and skill. But, no mechanisms had been provided for the technician to 
interact with the expert system, other than in trivial ways. No mechanisms were 
provided for the technician to understand and track the expert system’s diagnostic 
process. 

The important thing about this study was that we were able to analyze how the 
system performed under various unanticipated conditions, such as underspecified 
instructions. This required judgement, because the technician had to use knowledge 
of the domain in order to interpret what was really required. Misinterpretations 
and mis-entry errors also occurred, as well as errors in the knowledge base itself 
(the machine would sometimes make a mistake due to an incorrect rule in its 
knowledge base). 

Adaptation to special conditions was also a crucial issue. For example, it might not 
be possible to carry out the expert system’s plan, because some assumption behind 
that plan was not true. For example, a tool was not available to do the requested test 
or the system assumed a particular device was operable, when, in fact, it was not. 
And therefore, the test requested by the machine could not be carried out until the 
device was at least minimally operable. 
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In summary (Figure 12) we found that there was a significant role for the 
operational people in this system. But the expert system, if anything, hindered 
human diagnosis. It did not even provide a memory trace of all the diagnostic tests 
that had been run up to that point in time on the system. The operator was 
functioning without any aid whatsoever, even as simple as a electronic notebook. 
Yet he was forced to make a parallel, independent diagnosis in order to do the job of 
supervising and interpreting the information provided by the expert system. Again, 
the question is not whether the expert system did or did not help; the question is 
whether there is support for the human’s role. In this case, there was none. 


Expert Systems for Troubleshooting 

original %oal: reduce technician skill requirements to reduce costs and 
give management tighter control over technicians 

0 £iual resiifts: 

1) successful diagnosis required significant technician participation 
and skill ( almost 80% of test cases) due to unanticipated 

variability 

2) no mechanisms were provided for the technician to direct the 
expert system 

3) no mechanisms were provided for the technician to understand or 
track the expert system’s diagnostic process (black box design) 

4) some technicians discovered ways to manipulate the expert 
system 

5) technicians who passively followed the instructions of the 
machine made more execution errors 

6) technicians had to perform partial diagnosis on their own without 
any assistance or information from the expert system 


Figure 12 

LESSONS OF TECHNOLOGY CENTERED AUTOMATION 

In summary, Figure 13 lists the lessons learned when a technology centered 
approach to automation is employed. First, the human role in the system is changed 
in unforeseen ways. However, if the new system supports the new human role, the 
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person will be able to effectively accomplish the task. Second, there is a myth about 
de-skilling. Sometimes skills are reduced, but in general, skills are changed. The 
mix of skills, the kinds of people needed to operate a system, change. Third, the 
critical human role is to adapt to unanticipated variabilities and to amplify the 
requisite variety in a system. The person is there to help the machine do the job. 
Finally, new kinds of error forms and new kinds of system breakdown patterns can 
happen. The frequency and/or relative consequence of an error can be changed, and 
these new error forms, as well as their consequences and frequencies, need to be 
examined to be sure that everything has moved in a positive direction. 


Lessons of Technology Centered Automation 


shifts in automation have changed the human role in system 
performance in unforeseen ways 

de-skilling myth — changed pattern of skills ; it does not eliminate 
human skill 

critical human role is to adapt to unanticipated variability 
new error forms and types of system breakdowns 


Figure 13 

We can also summarize the fallacies in automation across cases like those discussed 
above: 

• incomplete automation where the machine is assumed to be fully 
autonomous when it is semi-autonomous; 

• failures to understand and support the human’s new or changed 
role; 

• brittle machine performance due to designer’s overconfidence bias; 

• pseudo-consultants; machine locus of control. 
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HUMAN CENTERED AUTOMATION 


In contrast. Figure 14 summarizes the lessons learned from a human centered 
automation approach. First, a human locus of control is required. The operator 
must have effective authority over areas of task responsibility and this cannot be 
merely lip service authority. The operator must have control of the machine 
resources including some degree of freedom of action and methods to instruct or 
direct the lower order machine agents. The machine should always provide support 
to the human. Designs requiring a choice between fully automatic or fully manual 
operation must be avoided. 


Human Centered Automation 


1. human locus of control: 

• effective authority as well as responsibility 

• control of machine resources, i.e., degrees of freedom of action 
including ways to instruct or direct lower order machine agents 

• the machine should always provide support to the human (e.g., 
avoid cases of forcing the human to chose between doing the task 
completely by himself or letting the machine do it completely by 
itself) 

2. human as supervisor: 

• monitoring of lower order agent 

• greater need for high level situation assessment 

• human tracking of machine state (what does it know, what 
is it doing, why is it doing it, what will it do next) 

3. support for error detection and recovery: 

• communication breakdowns between multiple agents 

• use machine intelligence for constraint checking and 

critiquing 

• support human situational awareness 


Figure 14 
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Second, the general role of the human supervisor in these highly automated systems 
is to monitor the activities of the lower order agents. This means there is a much 
greater need for high level situation assessment and new information displays to 
help accomplish this job. The need for human tracking of what the machine is 
doing, why it’s doing it, and where it’s going to go next is also greater. Third, the 
human role in error detection and recovery needs to be actively supported. 

DISCUSSION 

Question: Are you saying that automation should not be used at all? 

Response: No. As I said at the beginning, the issue I ve been working on is not 
whether or not to use technology. It will be used for a variety of reasons. The 
question is how is it used. Studies that I’ve done related to process control, as well as 
other studies, have all indicated that more leverage can be obtained from 
technology improvements and that some negative post-conditions or consequences 
of automation can be avoided if the effect on the human role is taken into account. 
Support mechanisms must be provided. It is not possible to say, in any absolute 
sense which is better — automation or lack of it. Rather, it is a question of the choice 
in level of automation. What post-conditions result for the human’s role and how 
are they supported? The total system performance should improve. The question is 
not whether the person alone can do a better job than the machine alone. I want the 
two together to do it better. 

Question: What do we have to change? What do we have to do to work around this 
problem? 

Response: We must be able to provide operational people like you with the tools 
(decision automation technology and information systems) to understand the 
possible error forms. Can we develop ways to use the technology to provide 
effective explanations of the basis for a machine-suggested diagnosis? My research 
and system development program with the process control industry is to provide 
designers with such tools. My mission today was not to go through these techniques. 
We do not by any means have all the answers, but we do have some. And I think 
together we can develop more. 

Comment: Some of the major points I learned from this presentation are. 

Our pilot training needs to be changed a little bit. We have to spend enough time in 
teaching the pilot what the designer expected the machine do when he designed it. 
This may sound like a cliche except that when we trained the pilot, we said, “push 
this button to go up, push this button to go down, but we have not spent a lot of 
time teaching how to do this manually, efficiently and safely. We should convince 
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the pilot that the designer of the flight management computer intended the machine 
to fly the aircraft exactly like the pilot would. If the pilot understood this, then 
when the machine is about to make a gross error, the pilot would recognize the 
gross error and be sensitive to the fact that the gross error is about to occur and will 
be able to correct it. 

Another place where our training has to be changed is the machine has to indicate 
the priority for fault management to the human. In the power plant example, the 
computer system could have worked if things had been prioritized. In some aviation 
cases, they are, but in other cases, a laundry list does not help you at all. Now the 
EICAS system in the 767, for example, is good in some ways and not quite perfect 
in other ways. A list of yellow messages is fine as long as there are not more than 
about 4. As soon as there are more than 4, it is difficult to sort them out. To the 
designer, the answer may be simple — prioritize by indenting the less important 
ones. The problem is that for the human being, indentation means a subset of the 
primary set. So that does not work too well either. In fact, we are very vulnerable 
to EICAS misreading and there needs to be a better method of prioritizing the 
display. 

The last and most critical thing is we have to teach the pilot that it is not a question 
of either all automatic or all manual. We have to teach him when to interrupt the 
machine’s action in a supervisory manner. It should not be an all or nothing 
approach but somewhere in between. In other words, the crew may need to 
eliminate the FMS and use the remote control panel, but they may not need to 
disconnect everything. That’s the direction our teaching must take if we are to help 
the human interact with the machine. 

Comment: Your examples in the banking industry indicated where they used to 
have problems across the board in the automation area. But these situations have 
been rectified. It seems to me that the technology has now reached the point where 
we’ve got to take the blinders off of the engineers and designers so that they 
understand that the technology does not stop at the front of the display. The 
problem is how to display the information. The technology problem goes right to 
the actual application and that’s where all the problems are. Designers were only 
concerned with how to generate information contained in the computer and how to 
get it to the controller’s CRT. The designers then incorrectly assume that their 
problem is finished. The controller/pilot must not be left to figure out how to use 
this information. 

Question: Sometimes it’s difficult to get a list of examples of what not to do. Do you 
have a list, even a small list? 
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Response : Figure 14 is an attempt at this list. A variety of cases would be required 
to develop a more thorough list, but it should now be possible to put such a list 
together. 

Question: Dave, it seems to me that problems occur in trying to design a totally 
automatic, safe system. Conversely, success happens when the automation was 
designed as a tool for the operator. We should stick with this philosophy in the 
cockpit. It should be designed as a tool for the pilot. Up to this point, I ve looked at 
the FMS as another navigation tool, an extension of what we had in the past. The 
new systems should not take the place of the pilot and that is the important issue. 

Response: I think you’re right. The problem researchers and developers like myself 
have is determining what it means to design the automatic system as a tool for an 
operator. And I think the down side is that lip service is given for that philosophy. 
Sometimes it seems as if we’re moving along in the right direction when in point of 
fact, we may be completely undermining the operator’s role in the system who may 
soon be only running alongside the automatics. 

Question: Dave, everybody wants guidelines. Do you believe that there will come a 
time when you can sit down and make up a list of general guidelines? 

Response: I think the answer is partially yes, but it would look very different than 
the traditional human factors guidelines. The problem is that the typical guidelines 
usually do not relate to the real design process or the work constraints which must 
be considered. Such guidelines are therefore irrelevant. 

Question: The examples you gave here, particularly for banking, illustrated how 
the automation can lead people astray which at least gets people thinking about it. 

Response: That is where guidelines will be useful. They may not be guidelines in the 
traditional sense, but they are pointers into relevant pieces of the literature. As 
examples, they function as reminders for designers. For example, something as 
simple as a computerized meter, a meter on a computer screen, doesn’t have to look 
like an analog instrument. It can have all kinds of interesting features, most of 
which you rarely see. The designer could use this part of the database to get a long 
reminder list of issues relevant to his application. 
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MIFTHANSA COCKPIT SYSTE MS SURVEY: A-310 

Captain Peter H. Heldt 
Chief Technical Pilot— Lufthansa 

INTRODUCTION 

Lufthansa is using cockpit surveys to obtain up-to-date information and feedback 
from their flight crews as a basis for cockpit specifications. 

1976-Survev 

In 1976 we conducted a survey of our existing fleet of: B-707, B-727-200, B-737- 
100, B-747-200, DC10-30 and A300-B2. At that time, development of modem 
avionic equipment indicated an imminent change in cockpit design. Questions 
regarding whether and how to continue with further automation on the flight deck 
were of special interest. The results of that survey were published shortly after 
introduction of the A3 10-200 and the B737-200 Adv. Both aircraft have Flight 
Management Systems (FMSs), a new generation of auto flight systems, and are 2- 
crew airplanes. 

This paper includes data only for the A3 10-200. 

Our A3 10-200 fleet consisted of thirteen A3 10-200 airplanes at the time of the 
survey “snap shot.” The A3 10-300 and A300-600 were not yet delivered. Vertical 
navigation was only in an introductory stage with so-called OP++ status. Thus 
questions regarding VNAV could not be included. 

The Questionnaire 

The questionnaire was anonymous. The only personal data requested were: 

Function (CM1 or CM2) 

Types of airplanes flown previously 
Hours logged on A3 10. 

All questions were to be answered with standardized ratings on a five-point scale 
with a neutral center position. Free text comments were encouraged. The entire 
volume of documented written comments represent a valuable source of 
information for system designers. The questionnaire consisted of two main parts. 
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Part 1: Cockpit lay-out, general handling qualities and airplane sys- 
tems 

Part 2: Electronic crew interfaces: ECAM, EFTS, AFS, FMS 

Part 2 was subdivided into the following four main topic areas ac- 
cording to a standard man-machine interface model: 

I. Physical Interface (reach and see) — control location, reach and 
handling, display location, readability, color and lighting, etc. 

II. Interface Dialogue or Operational Considerations 
(understanding) — Ease of understanding of operational rules, display 
rules, interlocks, and amount and kind of required training. 

III. Interface Tools (usability) — General usefulness, adequateness 
and importance of features. 

IV. Organizational Aspects of the interface (appropriateness in the 
operational environment) — Factors like reliability, logistics, ATC- 
constraints, etc. 


Some Statistics 

Total number of questions per questionnaire = 654 
Number of questionnaires distributed = 202 
Number of questionnaires returned =121 (60%) 


ECAM (ELECTRONIC CENTRALIZED AIRCRAFT MONITORING) 

General Observations 

Pilots with more than 500 hrs on the A3 10 gave more positive ratings than those 
with less than 500 hrs. 


Physical Interface (Reach and See) 

Generally, the physical interface was judged positively. The new visual display 
units are welcomed. Only a few aspects were criticized: 
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Contrast and Brightness 


Contrast and brightness of the displays were judged as inadequate under daylight 
conditions. CRT surfaces often are contaminated by dust and fingerprints. Display 
wiping and cleaning is a common preflight procedure. 

Interface Dialogue (Understanding the ECA M1 

Automatic sequencing, display rules, and readability all received positive ratings. 
However, the aural warnings were mentioned as being too loud. All pilots attest to 
their effectiveness as “attention catchers,” but the comments of the pilots differ 
regarding the issue of direct failure recognition via specific tones. Sixty-four 
percent of the pilots state that the present number of different aural warnings is 
“just right.” (The A3 10 has 7 aurals plus voice warning. The A300-B2/B4 had 13 
aurals + voice, which received a clear “overdone” in our 1976 survey). 

Learning to operate and understand the ECAM does not seem to represent a 
problem during normal operation. 

Interface Tools/Support of the ECAM 

The principle of computer-aided guidance during normal and abnormal operations 
is generally welcomed. The general ECAM performance to cope with system faults 
is judged to be of value by 87% of the pilots. The system displays are judged to be 
important or very important by 97%, while only 21% of the pilots find the 
dedicated warning light display panel (WLDP) of any value. 

This favorable overall rating however, also includes some comments regarding 
ECAM’s weaker elements: Checklists offered by ECAM received mixed ratings. 
The comments mainly center on the idea that the checklist design sticks too much to 
the 3-crew basic A300-B2/B4, i.e., the procedures involved to operate the aircraft 
systems are basically unchanged. Pilot actions are sensed by push-buttons, and this 
calls for some additional activity ( “monkey switching”). 

Because of limited redundancy and computer capacity the ECAM system requires 
the pilot to change from screen to paper checklists and vice versa. This is perceived 
as being confusing. 99% of the pilots state that thorough training is required to 
handle this aspect. 

Generally, pilots felt that proficiency in dealing with abnormal situations cannot be 
achieved during line operation. Therefore, they uniformly called for more 
simulator or refresher training. 
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Organizational Interface (Fit into the Environment) 


The survey results showed that reliability, maintenance and logistics do not seem to 
represent any problems. Ninety-Five percent of the pilots indicated that they have 
never or have seldom seen an ECAM failure, nor did they have difficulties in 
getting spares when needed. 

Many pilots claim that the cruise phase, where most T/O-inhibits are cancelled, 
begins too early (1,000 ft/GND). In dense traffic areas the crew is still busy with 
departure procedures. 


EFIS (ELECTRONIC FLIGHT INSTRUMENT SYSTEM) 

General Observations 

As with the ECAM more positive ratings were assigned to the questions by crews 
with more than 500 hrs. flight time. 

Physical Interface (Reach and See) 

Again, these interface aspects were judged overall very positive. Changing personal 
instrument scanning technique from electro-mechanical instruments to CRT- 
displays seemed to present no problem. The following minor complaints should be 
considered: 

• Vision and cross cockpit readability of LCDs for decision height 
and flight path vector is reported to be only marginal. 

• The location of the navigation display (ND) which is obstructed by 
the control column is not optimal. 

• The quality of the artificial voice for radar height is rated as good 
or very good by 94% of the respondents, but our pilots state that it 
is too loud. 

• As with the ECAM, the EFIS displays require frequent cleaning to 
reduce any dust or reflection which is reported to be annoying 
during daylight operation. 
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operational Interface 
fl Inderstanding EFIS D isplay Formats) 

• Our pilot ratings substantiate that these new visual display units are 
superior to electro-mechanical flight instruments and are self- 
explanatory (intuitive) to a large extent. 

• The detailed speed scale obtained an excellent rating; 94% said it is 
advantageous. 

• The display of preselected altitude in the primary flight director 
(PFD), which is just a duplicate of the glareshield selection, 
triggered a request for a “full blown” PFD presenting all basic T 
data. 

• The indication of radar height in the PFD consists of a plain 
numerical readout, but is supported by artificial voice. Eighty- 
three percent of the pilots confirm that their awareness of radar 
height and rate of descent is satisfactory. This implies that aural 
signalling for essential flight parameters is adequate and acceptable. 

• On the other hand, presentation of drift angle is flawed: 71% say it 
is poor or very poor. 

• Pilots did not report encountering any problems with EFIS during 
training. Line and simulator training are considered most efficient 
and valuable, while using the airplane operations manual (AOM) is 
considered less efficient. 

The EFTS as a Tool (Support of the Pilot) 

Again EFIS are generally accepted. Some details should be mentioned: 

• The design of the speed scale again received high marks. 

• 95% of the pilots say that their speed awareness is helped. 

• The standby airspeed indicator is thought to be rarely used for 
reference. 

• The flight path vector (FPV) received less approval. This new 
display element needs further development. Some pilots want a 
FPV with flight director (F/D) commands. 
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• Among the various ND Modes, the MAP Mode is judged to be the 
most important one, followed (by a considerable distance) by the 
PLAN, ROSE and ARC mode. 90% of the pilots state that the ND- 
CRT-display is an eye-catcher. 


AUTOFLIGHT SYSTEM 
Physical Interface (Reach and See) 

The similarity of the FCU control knobs in the glareshield encourages errors. 
Twenty percent of our respondents mentioned that the knobs are to 
indistinguishable from one another, particularly due to their close physical 
proximity, thus this design is likely to induce mistakes. A typical comment states: 
“all in a row look pretty good, but it takes time to identify.” 

The legibility of the labels is good under daylight conditions; at night, legibility is 
impaired. Lighting level of different panels is not synchronized. A lot of comments 
criticize the PRESET SPD/MACH indication and label. 

Dust behind the glass window of LCDs hampers readability. 

Eighty percent of respondents find it important that most FCU parameters are 
repeated in the PFD and ND or are summarized as flight mode annunciation 
(FMA). However, 51% report having missed a repetition of the vertical speed (V/S) 
value and the SPD/Mach value in the PFD. 

AFS Operational Interface (How to Use 10 

The multiple functions for the SPD knob seem to be acceptable. This does not hold 
for the altitude selector (30% critical reports). Whenever rotary push/pull knobs or 
simple push-buttons are intended to perform the same function, push-buttons are 
preferred. 

There are a variety of comments on the preset/SPD/Mach function and its 
readability: 

• The preset function is required but actually operating the button 
was judged to be too difficult. 
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• The rotary knob for the V/S is strongly criticized by 47%. Occa- 
sionally the knob is turned in the wrong direction. 

• 42% have experienced inadvertent changes of dialed selections 
when operating the selector knobs. Some comments mention that 
this is the reason why push-buttons are preferred for “engage’ 
functions. 

• The information of the FMA in the PFD regarding color, quantity 
and arrangement seems to be highly appreciated. 

• 74% of the crews use 100% of all AFS functions. Every second 
copilot indicates a desire for additional functions. Captains seem 
just happy with the modes provided. 

• There is a common request for a “*” symbol for speed and altitude 

in profile mode in order to see what the A/P is going to do next. 

Crews report that throttle movement often is the only cue they have 
to know that a mode change has occurred. 

• Line training (learning by doing) is regarded as the best means of 
instruction for the AFS, according to 83% of respondents. The 
simulator is accepted by 1/3. The AOM is used by only 19%, 
whereas 49% consider manuals a poor training method, and 30% 
report never using the book. 

AFS as a Tool in Da ily Operation 
(How Well Can a Task be Performed) 

The number of modes for lateral, vertical navigation and thrust control are 
considered adequate by 83% of the respondents. 

Use of the AFS 

T/O: 75% use AFS (thrust only) plus FMS in NAV + profile (PROF). 

11% report performing the T/O manually with FD. 

Climb: 91% report using all features, only a small group reports 

flying manually. 

Cruise: AFS is uniformly reported to be used almost 100% of the time. 
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Descent: No uniformity here, although the largest group uses AFS + 

FMS in NAV. Profile descent speed is reported to often be too slow for 
ATC requirements. 

A pproach: 55% report flying the approach manually with ILS. Non- 
precision approach is flown manually (39%). Others report using 
AFS+FMS in NAV. Visual approaches are flown manually without FD 
(78%). 

Landing: Landings are performed manually by 95%; control wheel 
steering (CWS) is reported to be almost never used. 

The guidance quality of the AFS in lateral modes is judged to be accurate, but in 
LAND mode the ILS capture of the system is criticized. Comments report 
overshoots when capturing. Also, during altitude capture, oscillations are reported 
to occur. Throttle-movement is judged as too active during certain flight phases. 

Comments also note that synchronization of both engines by the auto throttle system 
should perform better. 

The AFS is sometimes switched off for passenger comfort reasons. 

Or ganizational Interface: fFit into the Environments 

Eighty-five percent of respondents feel that ATC requirements can be met with the 
present system. Seventy percent indicated they rarely encountered missing 
functions due to lack of spares or maintenance problems. 

• CAT HI is reported to almost never be impaired by AFS malfunctions. 


FLIGHT MANAGEMENT SYSTEM (FMS) 

Physical Interface: fReach and Seel 

• Cross cockpit reach of the opposite CDU is reported as an incon- 
venience. 

• Readability of the keyboard is judged to be good under daylight 
conditions but poor during night flight (53%), due to dimming 
performance. 
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Size of the keyboard is criticized by 1/6 of the pilots. 

The arrangement of the keys is accepted by the pilot group with 
more than 500 hrs. False inputs reportedly do occur however (keys 
are too close to each other). 

The parallax of the “line select” keys fosters inadvertent inputs. 

22% judge the FMS CRT as too small and too overloaded to locate 
the desired information quickly (graveyard of numbers). 

65% ask for a color display. 

Operational Interface: (Working with the FMS) 

The FMS menu structure and the scratch pad operation is much 
liked. 

Rigid input format rules frequently produce “incorrect data” mes- 
sages. 

As is the case with the AFS, learning by doing seems to be an ap- 
plicable principle for the FMS. 

FMS as a Tool 

Pilots report that in some situations it is difficult to find specific 
data quickly, (although these situations are relatively rare). Typical 
cases are: return to departure, route change, or waypoint change. 

The response time or computational performance of the FMS is 
judged to be far too slow. 

Lateral Nav is judged outstanding while the available vertical NAV 
is just average (note: no profile (PROF) existed in 1986). 

Due to overall satisfaction with the FMS, aircraft control is gen- 
erally delegated to the FMS or FCU. During approach and landing 
phase, the FCU is preferred in order to stay head up. 

Inputting navaids which are not in the current database and entering 
waypoints via the scratch-pad is judged to be workable without 
undue hardship. 
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Bearing/distance to a waypoint is judged highly important. 




• SEC FPL (secondary flight plan) is judged important by 78% of 
respondents. 

• Misaligned maps occasionally occur and are annoying when they do 
happen. 

The following additional items are strongly desired features (should be 
given high priority): 

display of minimum safe altitudes, 
engine out departure procedures, 
airport layout. 

CONCLUSIONS 

Overall, our pilots like automation. However, flying with the automatics must be as 
good or better than flying manually. Some problems do occur with automation. 
“Keeping the pilot in the loop” is a mandatory requirement for any automated 
function. FMS and ECAM are both well liked. However, both systems are not yet 
optimally designed. Initial development of the FMS was promoted and tested by a 
relatively small group of pilots. Further development should be based on a broad 
(international) range of airline experience. Advanced flight management systems 
must incorporate an improved crew interface, higher computational performance, 
and a better fit to the ATC-environment. 
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Flight Deck Automation: Promises and Realities 
Final Report of the Working Group on 
AUTOMATION AND AIR-GROUND COMMUNICATION 

Compiled by Renate Roske-Hofstrand 


Chair: Jack Wisely, TWA 

Vice Chair: Renate Roske-Hofstrand, NASA-Ames Research Center 
Members: Steven Alvania, FAA 

James Danaher, NTSB 
Alden Lemer, FAA 

George Steinmetz, NASA-Langley Research Center 
INTRODUCTION 

This report is based on discussions held at the NASA-Industry workshop entitled, 
Cockpit Automation: Promises and Realities, held in August 1988 at the Carmel 
Valley Inn, California. Several themes emerged during the working group 
meetings. Among the main themes are: 

1) Because controllers and pilots cooperate to achieve safe, orderly, 
and expeditious traffic flows through the National Airspace 
System (NAS), designers of automated systems must consider the 
tasks and goals of both the pilot and the controller. 

2) The primary domain for air-ground communication involves 
navigation on the pilot’s side and traffic flow management on the 
controller’s side. Flexible, well designed communication 
interfaces must be developed where sharing of information 
between controller and pilot is easily accomplished without 
increasing workload levels. 

3) Cockpit automation development must evolve so as to achieve 
compatibility with the NAS efficiency goals which relate to 
increasing traffic density in available airspace. Cockpit situation 
management must be matched to the traffic flow management 
concerns of the controller. 

It should also be noted that the air/ground working group was tasked with 
discussing some of the current issues in air/ground communication and therefore a 
conscious effort was made to avoid any detailed discussion of topics explicitly 
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related to future planned datalink technology during this workshop. The 
overwhelming feeling of the working group members, however, was that it is in the 
context of these future systems where new approaches to design are called for and 
can be of most benefit A follow-up workshop on these issues was suggested. In fact, 
it is anticipated that the interdependencies noted below between pilot and controller 
tasks will be even more closely coupled in the future and will likely include other 
ground-based elements such as airline dispatch interfaces and/or airline 
maintenance operations. 

TOWARDS COORDINATED DEVELOPMENT AND DESIGN 

The goals of automation in both the cockpit and control room environment have 
traditionally been defined in isolation from one another. Because these 
environments both function within the NAS, there exist strong interdependencies 
which need to be taken into account in the definition and design of automation tools 
for both pilots and controllers. 

The interface between pilot and controller primarily supports communication 
activities regarding an aircraft’s safe progress through airspace populated with 
many other aircraft. A high degree of cooperation between pilots and controllers 
and adherence to commonly understood (standard) procedures is the foundation for 
current safe operations. 

It is important, however, to realize that the specific task goals of the pilot and the 
controller differ in the following way: Pilots are in command of their aircraft only. 
They are concerned with navigating through traffic and weather in their immediate 
vicinity. Therefore, pilots can be said to be single-aircraft centered in their tasks 
and goals. 

Air traffic controllers, on the other hand, support the pilot’s task of safe navigation 
but share their attentional resources among all aircraft in a given sector for which 
they carry responsibility. Controllers are increasingly concerned with system-wide 
efficiency as it relates to traffic throughput and total number of aircraft served. 
Controllers can be said to be multiple-aircraft centered or “distributed” in their 
tasks and goals. 

How does one then appropriately shape and coordinate the development of 
automation tools in this interdependent context? Cockpit automation development 
must be able to specify what the consequences of a particular interface design are 
with respect to both the pilot’s tasks and the controller’s tasks within the operational 
backdrop of the overall system. 
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SYSTEM-WIDE CONSEQUENCES OF COCKPIT AUTOMATION 

The advent of automation in the cockpit (i.e., Flight Management Computers, 
global navigation systems, sophisticated autopilots, etc.) assists pilots in complying 
with standardized procedures and controller requests. With automated on-board 
navigation equipment complex navigation and route structures can be adhered to 
more precisely (as long as no unforeseen changes occur). Conceptually, at least, the 
advent of cockpit automation should be beneficial to controllers as well as to pilots. 

Automated navigation usually results in more precision and predictability of an 
aircraft’s position in the available airspace since maneuvers are executed 
automatically and do not incur the common costs of pilot response or deviation 
from prescribed routes or altitudes. Additionally, complex route structures with 
many constraints (minimum or maximum altitudes or even time limits) present no 
additional burden on the pilot, because they are executed by the on-board 
computers. 

If aircraft, because of their cockpit avionics, can adhere more reliably to complex 
navigation structures, then controllers could have more flexibility in using such 
structures and issuing clearances accordingly. Furthermore, controllers, in 
addition to having a larger repertoire of possible alternative routes available, can 
expect more consistent adherence to their instructions. A controller s expectation 
about the possible range of deviations from the issued clearances is positively 
affected, i.e., possible flight path deviations are minimized since navigation is 
automated. This reduces the need for the controller to re-check and continuously 
monitor an individual aircraft’s progress since he/she can rely more strongly on the 
execution of a particular flight path over time when dealing with an “automated 
aircraft. 

Cockpit design at present occurs in isolation from the larger system context. For 
example, the fact that controllers now have no explicit information available to 
them regarding the type of automation equipment available on a particular aircraft 
precludes their ability to make strategic use of this information. In other words, the 
benefits of cockpit automation cannot be fully realized for the system as a whole. A 
question that needs to be considered in this context is the following: Should a 
controller’s strategy for dealing with traffic consider the equipment available on 
board the aircraft? Undoubtedly the answer for the future must be affirmative, i.e., 
the pattern of control and cooperation between pilot and controller must evolve 
from common, basic assumptions about an aircraft’s capabilities. 
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TRADEOFFS: PILOT VERSUS CONTROLLER CONCERNS 


Flexibility in route assignments is necessary to move aircraft safely and efficiently 
through the available airspace. A heightened emphasis on traffic “throughput” 
makes strategic flexibility in traffic control a necessity. Pilots currently perceive 
increased effects of the NAS demands primarily as increased workload in the flight 
phases occurring in terminal areas. To a large degree this occurs because of poor 
interface design. Pilots in high density traffic areas must be able to respond quickly 
to changes in ATC clearances. Older technology aircraft fare better under these 
circumstances, presumably because the pilots themselves are processing and 
integrating rapid changes in ATC clearances, and they do not need to engage in the 
extensive re-programming efforts now required to keep the on-board systems of 
the “advanced cockpits” informed of these changes. Thus pilots of non-automated 
aircraft do not experience a noticeable increase in the level of workload as a result 
of ATC instructions. 

This paradox is also in part a direct result of control policies that deny controllers 
the use of different control strategies for automated aircraft. The mixed traffic is 
presently dealt with as if there were no performance differences among the 
differently equipped aircraft. While cockpit automation should allow for more 
flexible controller strategies in order to meet the ever increasing capacity demands 
in addition to reducing the pilots workload during critical flight phases, just the 
opposite occurs. Pilots of “equipped” aircraft experience increased workload levels 
as a result of changes in controller issued clearances. 

There exists, at present, a tradeoff between overall NAS efficiency, i.e., flexibility 
of control strategies, and the pilot’s ability to accommodate this flexibility in the 
automated cockpit. The potential benefits of cockpit automation are clearly 
compromised by the seemingly sluggish and isolated (from cockpit automation 
development) air traffic control system. The solution to this state of affairs rests 
with a joint commitment by airframe and avionics manufacturers, air carriers and 
the FAA to develop air/ground automation tools that are explicitly designed to be 
compatible with each pther. 

DIRECTIONS FOR RESEARCH IN INTERFACE DESIGN 

Among the specific complaints of pilots with respect to air/ground coordination are 
the lack of adequate electronic map displays. The anecdotal evidence presented at 
the working group points at both, an inadequate database in terms of completeness, 
and at the inadequate current format of the interface to the database. 
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The entire concept of an “image assisted communication system” seems to have 
eluded designers of current advanced cockpit systems. When the controller issues 
clearances the pilot should not have to resort to paper charts to follow the 
instructions. The electronic map display should be able to allow the pilot to easily 
locate unpublished temporary fixes and provide an interface with which controller- 
issued clearances for route structures can be stored and followed easily. 

Questions of maintenance and integrity of the navigation database need to be 
addressed. The potential benefits of a common pilot/controller geographical 
navigation database should be explored and evaluated. Development towards these 
jointly considered automation systems should be evaluated against the following 

guidelines: 

1) All aircraft in a given airspace can operate efficiently i.e., in a 
safe, orderly, and expeditious manner. 

2) The controller can maintain a high degree of confidence that an 
aircraft will follow issued instructions. 

3) The pilot has available suitable information displays to comply 
with controller-issued instructions. 

4) All humans in the system are comfortable with resulting work- 
load levels. 

5) Cooperation between pilots, controllers, and dispatchers is en- 
couraged and enhanced via easy to use, image-assisted communi- 
cation interfaces. 


SUMMARY 

Some major issues discussed by the working group are summarized in Figures 1 
and 2. Figure 1 illustrates the coupling of computer processing and ease of using the 
data. Figure 2 is a diagram of the information flow in a shared environment. The 
exchange of information via voice-link, or in the future data link, is most critical. 

Operational safety and efficiency goals demand full and adequate situation 
awareness from both the pilot and the controller. While pilots are single-aircraft 
centered in their tasks and goals, the controller’s attention is distributed over 
multiple aircraft. Hence there exist different needs which must be accommodated 
by the interface, i.e., the joint concern for an aircraft s safe navigation must be 
supported by features in the interface which are designed to support mutual 



understanding and effective communication. The inherent trade-offs between pilot 
and controller goals must be recognized and explicitly addressed. 

Designers must specify the consequences of their design decisions in terms of the 
communication interface between the aircraft and ground-based air traffic control. 
Future displays for the controller need to include information regarding the 
“automation status” of an aircraft and prediction functions for traffic paths which 
allow him or her early conflict resolution planning. Well-designed displays for the 
pilot must include easy to manipulate and easy to comprehend information 
regarding area navigation and descent profiles, possibly including superimposed, 
integrated weather information. Additionally, on the same display, the pilot could 
be shown traffic that is directly relevant to his aircraft’s flight path. The overriding 
goal for the design of controller and pilot displays must be a shared frame of 
reference for air/ground communication based on common geographical databases 
which permit a high degree of cooperation towards meeting NAS efficiency goals. 




Automation and Air-Ground Communication Working Group 
Carmel, CA; August 1988 


Figure 1 
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Figure 2 
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INTRODUCTION 

The design and implementation of increasingly automated systems on the flight 
deck raises a variety of potential human factors issues relating to the work that 
individual crewmembers perform. In addition to these concerns, however, there 
are issues which affect the crew as a whole; that is, the way in which crewmembers 
coordinate their activities together. The most obvious, direct effects include 
changes in task structure, changes in the interpersonal aspects of traditional and 
standard procedures, and changes in information flow and communication chan- 
nels. 

There are also indirect effects (i.e., effects which are less specifically tied to flying 
the aircraft). These include changes in the organizational structure of the crew 
which can potentially create shifts in authority and the redistribution of 
responsibilities. Whether company policies and training programs mirror such 
changes is a further consideration. Finally, there are indirect effects which are 
related to the problems of transition from one technology to another, such as the 
fact that proficiency must be maintained in manual backup systems in addition to 
partially- and fully-operating automated systems. 

The direct and indirect effects of automation listed above structured the working 
group discussion of major issues, but once the issues were brought to bear on 
specific operations, an immediate decision was made to discuss normal an 
irregular operations separately. Because problems and the strategies used to cope 
with each are quite different, we. also generated separate recommendations. 
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NORMAL OPERATIONS 


Effects of Aut omation on Task Structure 

In considering task changes from non-automated to automated systems (types of 
tasks, distribution of workload, prioritization in normal operations), we first broke 
tasks down into Pilot-Hying (PF) vs. Pilot-Not-Flying (PNF) roles. We did not feel 
that the Captain (CA) and First Officer (FO) roles and responsibilities had been 
altered, but that the PF/PNF task structure had changed considerably. 

In general, in the automated cockpit, the PF (regardless of whether this is the 
Captain or First Officer) has less direct control of the flight path, though potentially 
more precise control, and must assume a greater managerial function. The PNF 
who previously provided PF backup by waiting much of the time, and talking to air 
traffic control (ATC), has become a more integral part of the PF’s flight control 
duties. In more automated systems, the PNF must provide a type of backup which 
requires greater active participation in flight control and less systems monitoring. 
In particular, flight path control is filtered through a communication chain that 
involves both verbal and digital control display unit (CDU) inputs. For example, 
one carrier’s procedure for a simple altitude change requires both PF and PNF 
participation; PF to command a CDU entry, and PNF to change the altitude setting 
input into the flight management system (FMS). The airplane climbs or descends 
appropriately or inappropriately and both pilots must observe carefully in order to 
avoid gross errors of reception or data entry. 

Systems operations: Largely a non-problematic area, the changes in 
systems operations include a shift to more passive monitoring (normal 
operations only); hence a decrease in workload related to monitoring 
and control of subsystems. Although error messages via electronic 
crew alerting devices such as the engine indicating and crew alerting 
system (EICAS in the B757/B767) may occasionally be an annoyance, 
these are not major problems in normal operations. 

Primary flight control: A great number of changes are associated with 
primary flight control, and these issues will be discussed separately 
from navigational issues. Many flight path functions, such as 
horizontal path control, are non-problematic, particularly in low 
workload phases such as cruise. Vertical path control, however, is 
affected both positively and negatively. Unfortunately, the times of 
greatest benefit (airport traffic area and other high workload phases) 
are also the times when some of the negative effects emerge. For 
instance, ATC can issue a directive that makes vertical path adjustment 
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necessary. Although FMS operations, in general, create less of a 
demand on mental arithmetic, vertical path adjustment can be more 
difficult simply because of the laborious CDU data entry required of a 
time-pressured PNF. In addition, feedback regarding these 
adjustments can be frustrating because control is less direct than in 
manual systems. A computerized system is more unpredictable simply 
because response time may be slow or variable. 

Thus, in spite of the fact that the FMS can provide greater vertical path precision 
(previous systems did not provide vertical navigational capability), more advance 
planning is necessary and greater PF/PNF coordination is required due to the 
possibilities of unanticipated changes. In addition, the vertical path display can have 
a mesmerizing effect on both pilots distracting them from other flight duties and 
forcing their eyes inside. Frequently this occurs at a time when extra external 
vigilance is required during climbs and descents. This may increase the risk of mid- 
air collision. Whether by command, standard operating procedures, or simply 
planning ahead, the PF must therefore assume a greater managerial role in order to 
ensure smooth PF/PNF teamwork. 

Less direct control of flight path operations also creates a need for a different type 
of cross-checking and monitoring. Error analysis can no longer rely primarily on 
immediate feedback via motor skills; rather, as in the case of cross-checking CDU 
data entry, and “reading” the mode control display, checking and monitonng 
procedures increase cognitive workload. 

Navigation systems. Similar to flight path control, navigation systems 
operation has, in general, benefited from increased automation. 
Enroute navigation radio tuning has largely become a flight 
preparation data entry activity, relieving both pilots of some degree of 
inflight workload. Specifically, the entire route is entered into the 
flight management computer on the ground, and the computer 
automatically tunes each radio enroute, permitting precise lateral 
flight path control. No further pilot action is required. However, the 
benefits degrade when flight plan changes are required, and these cases 
will be considered in the Irregular Operations section to follow. We 
are again mindful of the irony, that the greatest benefit of automated 
navigational parameters occur in the terminal area and this is very 
often the area in which flight plan changes are requested by ATC.* 
Finally, it should be noted once again that monitoring tasks are 

* Negative effects involving ATC are not intended to focus on flight deck automation problems 
exclusively. The fact that ATC procedures are not compatible with the newer automated systems on 
the flight deck is an important but separate issue (see Working Group report: Air-Ground Commu- 
nications). 
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affected. Because much of the flight planning can be accomplished 
prior to flight, there may be a decrease in situation (geographic) 
awareness, if monitoring is not suitably adapted to this change in task 
location. 

Checklists. More highly automated systems operations in conjunction 
with EICAS-type messages have allowed many of the routine checklist 
procedures to become more efficient (e.g., dark cockpit design where 
“no lights” means “no problem”). Two important benefits were 
discussed: (1) Checklists may be shorter; therefore, each item takes on 
increased significance and (2) it is possible to shift the items found on 
the “moving” checklist (checklist performed while the aircraft is in 
motion) to a “stationary” checklist (performed while standing still) 
which is a definite safety benefit. In support of the sterile cockpit 
concept, the elimination of unnecessary communication during taxi 
prior to take-off may be an important deterrent to runway incursions. 

Flight deck communication: Automated systems have changed the 
typical communication patterns within the cockpit in a variety of ways. 

First, as mentioned above with regard to checklists, electronic crew 
alerting devices such as EICAS have benefited the sterile cockpit 
concept because communication during this critical time has been 
minimized and therefore made more meaningful. Another clear 
benefit to crew coordination is that increased communication between 
pilots is required for error-free flight path control. Because both 
pilots participate in operating the FMS, greater communication, hence 
greater awareness of die planned flight path results. 

A more subtle example of how automation affects communication relates to the 
availability of information for both pilots. There is no question that the new 
automated graphic displays greatly increase flight deck information resources. It is 
also generally the case that these displays are equally available to both pilots. Given 
this situation, there is theoretically less need to share information by means of face- 
to-face communication. On the other hand, the availability of information does not 
automatically imply that both pilots are always attending to the same data, although 
it is tempting to ASSUME that they are. Thus, communication may seem to be 
redundant at times. However, there are also times when a false assumption would 
not be tolerable. For example, when mode changes are made, the mode control 
panel (MCP) is equally visible to both pilots. However, because we would not want 
to falsely assume that all changes were noticed, mode changes should be announced 
in spite of the fact that this might be viewed as redundant communication. 
Verification of mode changes might be alternatively solved by the addition of an 
annunciator signal on the attitude direction indicator (ADI) itself. 
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In short, automated systems do eliminate the need for some types of face-to-face 
communication and this can be beneficial. However, there are other times when 
communication may seem to be redundant but it would be incorrect and unsafe to 
make that assumption. Certainly in some situations, the possibility of error would 
be unacceptable and the formalized sharing of information may be warranted, as 
well as increased cross-checking procedures. For example, setting the mode control 
panel inputs to permit descent away from the altitude window setting is particularly 
dangerous since it can result in controlled flight into terrain. Operation in split 
mode during descent presents a situation where autopilot altitude capture engaged 
with autothrottles disengaged could result in a stall. 

Summary of Task Structure Changes 

• Decreased mental arithmetic 

• More cognitive, less motor skill 

• Less active systems monitoring 

• Increased cross-check workload 

• More evenly distributed workload PF/PNF 

• More flight path control coordination 
(formalized crew coordination) 

• PF more managerial function 

• PNF has increased CDU flight control participation but less 
systems monitoring 

Cultural Changes 

Again, we wish to reiterate that Captain (CA) and First Officer (FO) 
responsibilities have not changed; that is, the Captain still holds the final authority 
and responsibility. 

However, the FO as PNF is now in control of more information than formerly and 
the CA must modify his/her team management to accommodate to this change. 
There are two major areas which should be addressed. 


Redistribution of responsibility for traditional tasks : First, insofar as 
the tasks for PF and PNF are redistributed more evenly, task allocation 
should reflect these changes. More important, both Captain and First 
Officer must be equally proficient in handling the increased PNF 
responsibilities. In particular, both pilots must be well-practiced in all 
areas of automated systems operations; from handling the entry and 
operations of ACARS data to CDU entry in making flight plan 
changes, navigational changes and vertical path modifications. 
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Direction of information flow: In less automated aircraft, crew 
members were frequently able to accomplish their tasks 
independently. Given the greater coordination necessary to operate the 
FMS, however, lines of communication are created which represent a 
different flow of control. (Note again that this refers to a change in the 
transfer of information, NOT a change in authority structure.) For 
example, when CA is the PF, a simplified conceptualization of the flow 
of information follows [CA -*■ FO -*■ machine], where flight path 
control is being accomplished through the FMS and affected by CDU 
data entry. However, when the CA is PNF, the reverse is true: [FO -*• 
CA machine]. Since this sequence must flow in both directions, this 
reverses the traditional system in which the unilateral direction [CA 
FO], or simply CA and FO working independently was more common. 
It is important that manuals and procedures reflect these changes and 
that the PF/PNF terminology is supported in conjunction with the 
CA/FO role distinctions. 

Summary of Cultural Changes 

• More even division of responsibility between PF/PNF 

• “Role reversal” in terms of information flow 

• Increased responsibility of PNF 

Recommendations for Normal Operations 

• Training to address workload distribution 

• Formalized crew coordination 

• Formalized FMC checking process 

• Design changes to minimize entry errors 

• Procedures design to follow cockpit design 

• Use of plan-ahead procedures (take advantage of automation) 


IRREGULAR OPERATIONS 

It became clear to our working group, in the course of discussion, that none of the 
negative effects was really very serious in normal operations. Rather, increased 
automation on the flight deck has been successful in reducing workload, increasing 
the amount of informational resources available, and permitting greater precision 
in several technical performance areas. 
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Areas of concern emerged only when normal operations were interrupted or no 
longer possible. Since some of these times occur fairly frequently and are not 
“abnormal” in terms of malfunction, we defined “irregular operations” as 
unanticipated deviation from intended operations with respect to a range of possible 
events. At the least extreme end of the range, irregular operations could be 
instigated by internal events such as minor system malfunctions or external events 
such as unusual ATC requests or new weather conditions. At the high end of the 
range, irregular operations would be brought on by hard failures which prescribe 
the use of formalized, written procedures (e.g., loss of pressurization or AC 
power). 

Minimum equipment lists : The puipose of minimum equipment lists 
(MELs) has not changed in more highly automated aircraft. However, 
the criteria for designing MELs now need to consider the implications 
of degraded automated capabilities. When irregular operations require 
a decreasing shift from a fully-operating automated system, there is an 
accompanying shift in workload and task structure which must be 
relieved by the appropriate level of resources designated by the MELs. 

Task structure: At the onset of “irregular operations,” the degree to 
which tasks must be reassigned will depend on the severity of the 
problem and the degree to which the automated systems will need to be 
shut down. In almost all cases, however, the PNF will change from 
being a passive systems monitor to an active systems controller. Other 
task reassignments would need to be based on the particular 
“irregularity” although there should be pre-determined rules 
(supported by company policy and training) that provide guidelines 
for switching from a fully operating automated system to a partial 
system or a total reversion to manual control. It was felt in general, 
that (1) there should be no arbitrary reversion and (2) the highest level 
of automation should be maintained, where possible, consistent with 
safety and tasks required. For example, because vertical path 
constraints and manual approach building are not in the database, these 
operations in a completely automated system result in an increase in 
the number of tasks and workload and such an increase might not be 
tolerable during “irregular operations.” These kinds of considerations 
must be weighed in selecting the level of automation that can be 
realistically and safely maintained. 

Pre-established priorities: Priorities for the task reassignments 
required by irregular operations also need to be pre-established and 
supported by company policy and training. Embodying the notion of 
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“A” tasks and “B” tasks where tasks are unambiguously prioritized, the 
switch from normal to irregular operations should be associated with a 
comparable task priority list as well. As an example: PF will fly the 
aircraft and handle ATC communication while PNF handles secondary 
communication and becomes active systems controller. As systems 
controller, PNF will either initiate a procedural worklist (irregular 
procedure checklist) or begin a situation assessment of the system. The 
PNF must be able to invoke rules for interpreting the system of alerts 
and cautions during critical phases of flight. Whether “full mode” or 
“split mode” switching is used will create a need for different rules. 

Recommendations in Irregular Operations 

• Minimum equipment list design 

- accommodate the implications of degraded automated 
capabilities 

• Task reassignment 

- establish priorities 

- mles for assignment 

- determine level of automation operation 


SUMMARY 

In summary, there are four major areas in which crew coordination is affected by 
increased fight deck automation. To take full advantage of the benefits of 
automation, the following elements are essential in the coping strategy: 

1. There must be an increased emphasis on crew concepts in both 
training and operations areas. 

2. There must be better pilot-to-pilot communication during ALL 
phases of flight. 

3. There must be a complete understanding of tasks and responsibility 
for task accomplishment in the more highly automated 
environment. 

4. Aircrews must know when and how to transfer from automated to 
semi-automatic to manual operations as the situation dictates. This 
implies both systems and interface knowledge as well as alternate 
courses of action available when normal operations are interrupted 
or no longer available. 
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INTRODUCTION 

Since the introduction of the autopilot into aircraft more than a half century ago, 
automation has taken over many of the tasks which were once the exclusive 
domain of the human pilot. When machines control or perform tasks and pilots 
are relegated to a monitoring or supervisory role, questions arise as to the extent 
pilots need to understand these systems. Understanding a system means not only 
being aware of what it is (or is not) doing, but also knowing the reason for a 
system’s action and anticipating what the system is going to do next. In general, 
the need to understand a system is closely related to the need for intervention by 
the pilot if the system fails to operate as designed or, in the opinion of the pilot, is 
operating in a manner which compromises safety. 

This report is intended only as a summary of the discussions conducted by this 
working group. A comprehensive review of known or of potential operating 
problems with automated systems is beyond the scope of this report. Problems 
which exemplify more fundamental issues in training, design, or operating 
procedures are provided for illustrative purposes only. Likewise, the solutions to 
these problems which have already been implemented or are recommended 
should not be construed as exhaustive of possible alternatives. 

CURRENT OPERATIONAL PROBLEMS 

For current operational aircraft, problems involving the pilot’s understanding of 
automation can occur in two areas: flight path management and aircraft 
subsystems management. To date, automation of subsystems management does not 
appear to present a problem for aircrews. However, the current state of affairs 
may change as aircraft age increases and subsystem failures occur more 
frequently. 
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The more immediate problems are associated with flight path management. 
Examples in this category are usually associated with Flight Management Systems 
and related areas of computer control of the aircraft flight path. Problems of 
standardization of mode control panels are repeatedly cited by aircrews. The 
problem appears particularly acute in the area of status annunciation where 
confusion may arise as to what is and is not under automatic control. The 
problem of standardization of mode control interface design has been aggravated 
by the mixing of aircraft fleets resulting from the large number of recent airline 
mergers. Lack of pilot-system interface standardization increases the cost of 
training and increases the potential for pilot errors in line operations. 

Distinct from these problems of standardization in design are problems which 
arise from inadequacies in the pilot-system interface of an automated system. 
Errors can and do result when system status annunciation is unclear or 
ambiguous. For example, if a Flight Management System can engage an autopilot 
independently of an autothrottle, the system should make the pilot aware that the 
system is operating in this “split mode.” Lack of pilot awareness due to pre- 
occupation with other duties can result in changes of aircraft attitude in the 
absence of coordinated throttle inputs. If the pilot, for whatever reason, is not 
aware of the status of the system, intervention may well occur too late. 

Annunciation of gradual or “soft” failures of autoland systems is another example 
where pilot-system interface design is particularly important. With the aircraft in 
close proximity to the ground and configured for landing, awareness of an 
autoland system’s impending loss of function becomes critical as rapid and precise 
pilot intervention may be needed. As increasingly sophisticated automated systems 
are introduced in areas involving the operating limits of an aircraft, addressing 
the problem of soft failure annunciation will become more important. 

Closely allied to the problem of status annunciation is the need for pilots to 
understand the design intent of an automated system. Currently, this receives 
little, if any, attention in the pilot training process. The design intent underlying 
an automated system can often help the pilot understand what such a system can 
and can not do during actual operations. Windshear-induced autopilot hysteresis 
is an example. Sudden changes in the direction and speed of the wind can result in 
autopilot-induced pitch control oscillations. These excursions can be quite large, 
and if occurring close to the ground, catastrophic. Understanding system design 
should cause the pilot to disconnect the autopilot immediately. However, pilots 
may be reluctant to do so if they attribute capabilities to the system that it does 
not actually possess, e.g., that it has the intelligence to adjust to unusual 
operational conditions. Failing to understand the capabilities and limits of 
automated systems are persistent problem areas in operating such systems. 
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COPING STRATEGIES 


For current aircraft, coping strategies adopted to overcome operating problems 
with automated systems fall into three areas: training, operating procedures, and 
after-market design changes. As is typical with any design problem, training takes 
on a disproportionate role. Unfortunately, pilot training on automated systems 
has been recognized as being less than adequate in both areas of systems 
understanding and use of the system in line operations. Incorporation ot 
automated system operation into existing Line Oriented Right Training (LOF 1 ) 
is occurring, although the cost of this type of training is high. Alternative 
strategies of systems training employing computer-based training systems are also 
being considered. 

The second means being used to cope with automation problems is to change the 
procedures for using the system in line operations. For example, operating an 
autopilot separately from the autothrottle is now prohibited by some earners as is 
the operation of autopilots in windshear and severe turbulence. Most, if not all, ol 
these changes have resulted from an operational incident or accident. This tnal 
and error coping method is inherently undesirable as it may incur enormous costs 
in lives and property. Operating limits of systems should be clearly defined prior 
to line operation as should the potential of these systems for design-induced 
human error. 

Finally, after-market design changes can have limited use in ameliorating 
operating problems with automated systems. Improving status annunciation 
symbology and operating interfaces have some value in this regard. Improving 
Control Display Unit design to provide easier re-programming of the FMC is one 
such example as is enhancing the speed with which the FMC will accept pilot 
inputs. However, such changes are often difficult and expensive. In some cases, 
annunciation displays and associated interfaces that could enhance pilot awareness 
of an automated system’s status cannot be accommodated in the cockpit without a 
major re-configuration. 

ON THE TECHNOLOGICAL HORIZON 

Obviously, the time to address design issues is during the design process. 
However, aircrew training to understand and operate future automated systems 
will always be necessary and should be factored into the cost of operating any 
advanced system. Fundamental design and training philosophies for automated 
systems need to be established for future advanced technology aircraft if 
operating problems with these aircraft are to be avoided. Advances in computer 
technology will almost certainly result in even more complex levels of 
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automation than are currently available. The result will be increased demands on 
limited training resources. 

Issues that are on the technological horizon are varied and far reaching in their 
potential impact on pilot interaction with automated systems. Examples of these 
mclude the incorporation of ground-air-ground data link systems which will 
make possible the automation of clearance delivery to the FMC. Automatic 
warnings of altitude deviations and of descents below minimum safe altitudes, 
automated altimeter settings, and many other services are possible. Increasingly 
sophisticated flight control systems are also on the horizon including gamma 
flight path control,* envelope protection, and relaxed static stability. Airframe 
subsystem management will become more sophisticated with the introduction of 
intelligent systems and the use of decision aiding technology to facilitate failure 
diagnosis. Clearly, determining the extent of pilot understanding needed to 
effectively control automated systems will become even more important in future 
aircraft than it is today. 


RECOMMENDATIONS 

It is evident that the knowledge required to fly automated aircraft requires more 
than simply knowing which button to push and when. However, it is important to 
emphasize that the fundamental role of the pilot has not changed (but see 
Addendum). This role is made explicit in FAR 91.3: “The pilot in command of an 
aircraft is directly responsible for, and is the final authority as to, the operation 
of that aircraft. As part of this role the pilot has in the past, and continues to, 
perform the function of systems monitoring and flight path management. The* 
machinery by which an aircraft is operated does not fundamentally alter this role 
though it may simplify or eliminate the need to perform certain tasks. This leads 
to the working group’s first recommendation: That a philosophy of flight deck 
automation be adopted which assures that the pilot plays an active role in the 
management of the aircraft flight path and that any information which affects that 
management should be provided either in the training process or as an integral 
part of flight deck design. 


Secondly, the development of flight deck information management principles are 
needed to support the integration of automated systems in future aircraft. Know- 
ing what kind of information is needed, when it should be provided and in what 
form it should be presented are essential to the design process. Standardization of 
functional requirements for automated systems interfaces, particularly in mode 
control panel design, is also needed as are guidelines to minimize the possibility 
of design-induced errors on the part of aircrews. Such standardization require- 


* Editor's note: Either automatic or pilot control of aircraft inertial velocity vector. 
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ments will require a more active role on the part of regulatory agencies than 
currently exists. 

Thirdly, the integration of automated aircraft into the Air Traffic Control (ATC) 
system and the eventual automation of that system suggest that problems of 
aircraft-ATC integration will increase unless a comprehensive systems analysis 
effort is undertaken. A key element in that effort should be the delineation of the 
role and responsibilities of humans (pilots and controllers) in an automated air 
traffic control system. 

Finally, the training of aircrews of automated aircraft must emphasize the 
understanding of automated systems and how these systems can and cannot be 
used in line operations. The design intent underlying an automated system should 
be an important ingredient in training program development. 

ADDENDUM 

The definition of the pilot’s role in automated systems is not without controversy. 
It should be understood that others have taken the position that the role has been 
altered by the tasks, i.e., it is a role that is more passive, less manual, etc. It may 
be that these differing viewpoints result from focusing attention on different 
levels of the pilot’s task hierarchy. In any case, this issue undoubtedly deserves 
more extensive consideration than is possible within the scope of this report. 
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INTRODUCTION 

Airline pilot training provides the interface between transport aircraft and the 
pilots who operate them in day-to-day line operations. Training is obviously 
important and, despite the use of sophisticated simulators and other advanced 
t rainin g aids, it continues to be very expensive. While there is no disagreement 
within the aviation community regarding the importance of effective pilot 
training, there is considerable disagreement on the kind and amount of training 
that is required to enable pilots to operate new and different airplanes safely and 
efficiently within the aviation system. 

Today, there is a consensus among training experts, both within and without the 
industry, that regulatory requirements (and those training practices that are based 
solely upon them) have not kept up with advancing cockpit technology. It is not 
surprising that such training is not efficient and, perhaps because of an apparent 
excessive preoccupation with automation, it has not always been sensitive to the 
wide gamut of operating needs of the pilots who routinely fly these aircraft. 

There is, therefore, a very clear need to review all of the factors involved in 
contemporary airline pilot training. It is particularly important to review the 
training from a systems perspective because of the interaction of the many factors 
that are involved. 

Airline pilot training, even without the complication of advanced cockpit 
technology aircraft, is a very complex subject. Its complexity is exacerbated by 
such factors as some very basic differences in airline operations, the widely 
varying training resources of airlines that range from the established trunks to 
newly-formed commuters, the varying needs of pilots with a wide range of 
established skills and experience, a broad range of aircraft and aircraft systems 
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and an ATC system badly in need of modernization. In addition, all of these 
aircraft must be operated safely and efficiently in day-to-day line operations in a 
dynamic variable operating environment. It was not possible to fully cover all 
aspects of airline pilot training in the time allotted to us. 

Therefore, the Working Group did not deal with those specific areas nor with a 
host of traditional generic training issues such as the kind and level of fidelity 
needed in cockpit procedures or limited part-task trainers, the role of “motion” in 
instructional simulators with full motion capability, the optimum amount and 
kind of feedback for computer-based instruction (CBI), the design of training 
programs based on present regulatory restrictions, adapting training programs to 
the varying needs of a diverse pilot population, problems in “differences” 
training, the effectiveness of the variety of teaching methods and teaching devices 
currently available, etc. Because it did not deal with these and similar items, the 
Working Group does not mean to imply that they are not important. 

However, in order to take advantage of the current general industry consensus 
that reexamination of airline pilot training principles, practices, and regulatory 
procedures is sorely needed, the Working Group concentrated on identifying 
general areas it believes should be included in the current reexamination of 
airline pilot training. It identified seven general areas it believes should be 
included in such a reexamination. The seven general areas are listed below and 
will be discussed in the following paragraphs. 

1) Review of FAR 121 Appendix E (initial) and F (recurrent) 
training requirements 

2) Human factors training for “Decision Makers” in the industry 

• FAA 

• Manufacturers and their vendors 

• Airlines 

• Pilot associations 

• Airline trade organizations (e.g., ATA, RAA, and 
IATA) 

• International organizations (e.g., ICAO) 

• NTSB 

• Specialized training organizations 

3) Pilot training in automation 

4) The role of the manufacturer and its vendors in training 
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5) Standardization and simplification in automated aircraft 

6) Workload management in the 2 person crew automated flight 
deck 

7) The potential role of digital flight data recorders in training 


REVIEW OF FAR 121 APPENDIX E AND 
APPENDIX F TRAINING REQUIREMENTS 

The requirements specified in Appendix E (initial, transition, and upgrade flight 
training) and Appendix F (pilot proficiency checks) apply to all airlines operating 
under Part 121. They, in conjunction with the training requirements of FAR Part 
121 Subparts N and O, effectively control air carrier pilot training in the United 
States for all airlines other than those commuter airlines that operate under FAR 
Part 135. 

The Working Group believes that present regulations are not fully responsive to 
the technical and operational requirements of contemporary air carrier 
operations, and that a full review of FAR 121 training requirements is urgently 
needed. It fully supports the training objectives of the Administrator’s Task Force 
on Flight Crew Performance in this area and believes this subject should continue 
to receive a very high priority. 

In addition, the Working Group believes that the review process should be 
formalized to ensure that similar reviews are made periodically on a 
predetermined schedule or can be made in response to technological advances. 

HUMAN FACTORS TRAINING FOR 
“DECISION MAKERS” IN THE INDUSTRY 

Over the years there has been growing recognition that human factors should be 
considered a “core technology” in all parts of the air transport system including 
the air traffic control system, the design, manufacture, and operation of transport 
aircraft, and the regulation of these basic elements. All of these affect training 

requirements. 

The growing recognition of the importance of human factors, like so many 
aspects of this dynamic industry, has been evolutionary. Not surprisingly, its 
growth has not proceeded at an equal pace among all elements. The consensus that 
“human factors” should be considered a core technology on a system basis has 
been only recently achieved. 
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The Working Group believes at least two things are required to take full 
advantage of the human factors potential to improve the safety and efficiency of 
air transport operations: 

a. First, individuals responsible for decision-making at all levels 
affecting design, training, operations, and regulation (and this 
includes those with responsibility for the allocation of funds) 
should have training (or indoctrination) in human factors. This 
should be at a level that ensures awareness of the importance of 
human factors, and in particular, its relevance to air transport 
operations. The training (or indoctrination) for these decision 
makers should be of sufficient depth to enable them to recognize 
human factors needs within their area of responsibility, and to 
recognize the need for additional expertise in specific areas when 
such a need arises. 

Organizations with such responsibilities include the FAA, aircraft 
manufacturers and their vendors, the airlines, pilot associations, 
the NTSB, and specialized training organizations. 

b. Second, it is equally important that members of the scientific 
community interested or involved in air carrier operations 
receive sufficient training or indoctrination in those operations to 
ensure that their recommendations and research are responsive to 
real-world needs and problems. 

PILOT TRAINING FOR INCREASINGLY AUTOMATED AIRCRAFT 

[Note: All of the recommendations in this section apply to both FAR Part 121 and 
Part 135 carriers] 


Basic Airmanship Skills and Knowing 

The Working Group believes that the addition of sophisticated cockpit automation 
systems has not reduced the need for or the level of basic airmanship skills and 
knowledge which have traditionally been required of airline pilots. Therefore this 
discussion assumes that pilots transitioning to advanced cockpit technology 
aircraft already possess those skills and that knowledge. In addition, the 
importance of the extension and application of those fundamentals to the advanced 
technology aircraft should be emphasized in the early phases of both ground 
school and simulator training. General aircraft familiarization should always 
precede detailed instruction in automatic features. 
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Monitoring of Automatic Systems 


Effective monitoring of the operation of automatic systems is an increasingly 
important responsibility of the flight crew. The development of methods to 
increase monitoring effectiveness should be given a high priority. Cockpit 
resource management (CRM) courses that emphasize the importance of 
monitoring and the role and increased responsibility of the pilot-not-flying (PN ) 
are needed. In addition, the importance of monitoring activities should get 
greater emphasis in both training and checking activities. 

It is particularly important that transition training includes not only the operation 
of the automatic systems and their limitations, but also their “design intent. It is 
not reasonable to expect pilots to effectively monitor the operation of automatic 
systems without providing them a clear understanding of how the system they are 
monitoring is planning to accomplish its specific task. 

LOFT 

One of the major advances in the training of airline pilots during the past decade 
has been the development of line-oriented flight training (LOFT). However, 
because LOFT is still a relatively new concept there have been wide variations m 
both its use and in the quality of the training provided. 

Despite these difficulties, the Working Group believes there is a need for greater 
use of LOFT in initial training in order to better prepare pilots lor line 
operations. There was a weaker consensus that in recurrent training, the primary 
use of the simulator should be in a LOFT environment. It is important to 
recognize that recurrent LOFT can be conducted in the more elementary visual 
simulators as well as in Phase I, n, and III simulators. 

Cockpit Resource Manag ement (CRM) 

There is a growing consensus within the aviation community concerning a 
pressing need to improve cockpit management and cockpit crew coordination. 
Although a variety of CRM training programs have been developed, the possible 
need for CRM programs modified or tailored for the pilots of advanced cockpit 
technology aircraft has been essentially ignored. 

The. Potential Use of Home Computers in Training 

The sensitive and intelligent use of home computers to fulfill l training 
requirements and for voluntary self-instruction should be explored. While there 
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are obvious potentials for misuse, there is also a considerable potential for 
fulfilling the needs and desires of all of the parties involved — air carriers, pilots, 
and the FA A. Implementation, however, can be a particular challenge for air 
carriers and the representatives of their pilots. 

The Role of the Manufacturer and its Vendors in Training 

Determination of the general training requirements needed to enable pilots to 
operate new equipment safely and efficiently should be considered an integral 
part of the design process. Determination of training requirements at the design 
stage of any changes or updates developed subsequent to the original design are 
equally important. These requirements need not be, and probably should not be 
specific, e.g., at the SBO (specific behavioral objective) level, but should clearly 
indicate what the designer of the system believes the pilot should know in order 
to operate that system safely and efficiently. 


After the initial design and the inevitable compromises and tradeoffs inherent in 
the manufacturing process have been completed, it is a logical extension of this 
philosophy to have the manufacturers of transport aircraft and their vendors play 
a larger role in two important training areas. The first training area is in the 
development of the specific training objectives required to satisfactorily operate 
their products. The second training area is in the development of the training 
aids, techniques, materials, etc. needed to achieve those training objectives. 

STANDARDIZATION AND SIMPLIFICATION IN 
2PC AUTOMATED AIRCRAFT 

There is a great need for more emphasis on standardization and continued 
emphasis on the simplification of all aspects of the design and operation of 2PC 
(two-person crew) automated aircraft. It should be given a very high priority by 
both the manufacturers and the purchasers of their aircraft. This problem has 
been exacerbated by the increasing number of aircraft leasing organizations, 
airline mergers, consolidations, etc. that are a part of the contemporary airline 
scene. Different names for the same item, different procedures to operate 
essentially the same system, and different symbology to display identical 
information can create very real problems for the crews who have to cope with 
them. Unfortunately, this frequently happens under less than optimum conditions. 

This by no means should be construed as a restriction on the development of 
improvements in transport aircraft. However, it seems very clear that a great deal 
of the lack of desirable standardization in current aircraft has little to do with 
improvements in the aircraft, their systems, or their cockpit symbology. 
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For the same reasons, this emphasis on standardization and simplification should 
be extended to flight operations and equipment manuals, operating procedures, 
and checklists. This is primarily a responsibility of the airlines and, to a 
somewhat lesser extent, the regulatory authorities. It should be given a high 
priority. However, it should be noted that, particularly in the case of operating 
and equipment manuals, this emphasis on simplification does not imply that these 
documents should not contain the often detailed information and data required to 
fulfill their basic functions. 

WORKLOAD MANAGEMENT IN THE 2PC 
AUTOMATED FLIGHT DECK 

Air carriers are urged to take a new and creative view of flight crew workload in 
automated 2PC aircraft. 2PC operational procedures and checklists should be 
carefully reexamined with particular attention to the workload required to 
perform them. Many carriers have a strong history of 3PC operations and there 
is considerable evidence that their operation of 2PC automated aircraft does not 
reflect advances that have been made in cockpit technology and in our 
understanding of flight crew behavior. The problem has been exacerbated by the 
large number of flight crew members who have transitioned to these aircraft 
from a 3PC airplane. 

Properly developed LOFT scenarios can be used to illustrate heavy workload 
conditions and identify problem areas. For example, a prominent problem area in 
current operations, and one that present training (and/or procedures) has not 
dealt with effectively in many cases, is whether or not to continue to use 
automated navigation systems when ATC clearance changes are received during 
the final stages of an approach. In these cases, the important decision is often 
simply whether or not to continue to use automation and reprogram or to simply 
turn the automation off. If flight crew workload is further increased by 
inappropriate policies or procedures, the problem can be clearly identified during 
the LOFT exercise. 

Considerable flight crew workload can be created by the requirement to perform 
non-operational, but important, tasks at particularly inopportune times. For 
example, calls for passenger connections, meal requirements, wheel chairs, and 
other passenger service items can be more than just a nuisance to the flight crew. 
This is by no means a new problem, but is becoming more critical because of the 
proliferation of high-density operations. In many cases these flight crew tasks 
may be either further automated (as in the case of many ACARS functions), 
eliminated or reassigned. 
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THE POTENTIAL ROLE OF DIGITAL FLIGHT 
DATA RECORDERS IN TRAINING 

This is an admittedly controversial item. The Working Group believes considera- 
tion should be given to the system-wide use of digital flight recorders to identify 
areas needing training emphasis. It can also be used to identify those that are not 
creating problems. This is not a new idea. It has been used successfully by several 
foreign carriers with the sanction and complete cooperation of their pilot unions. 
The key provisions have been a clear recognition by all parties that the sole pur- 
pose of the program has been to improve the safety of their operations and that 
the rigid restriction on the use of this data has been honored. 

The sensitivity of the recommendation creates a formidable challenge for all of 
the parties involved. The challenge is to develop procedures that permit taking 
advantage of state-of-the-art technology for their mutual benefit. The ability of 
data from digital flight recorders to improve safety and, in some cases, to justify 
a reduction in training requirements and training costs requires a cooperative 
intelligent utilization of that data. If this can be achieved, problem areas can be 
identified early, and safety can be improved. The reduction of training 
requirements and therefore training costs is an additional potential benefit. 

SUMMARY 

Historically, airline pilot training has developed in a relatively piecemeal and 
unexamined manner. While a great many changes in the industry have been 
essentially evolutionary, the cumulative magnitude of these changes has created a 
pressing need to reexamine current operating procedures and training 
requirements in light of automation’s demands and in the opportunities it presents 
to the aviation community. 

The recognition of human factors as a core technology has been a major 
breakthrough. Another has been recognition of the need to reexamine our 
training needs from a total system perspective. The challenge, and it is a 
challenge to both the operational and scientific community, is to take full 
advantage of our new-found knowledge. 
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INTRODUCTION 

The objective of this working group was to identify the influences, both positive 
and negative, of cockpit automation on the occurrence and detection of error on 

the flight deck. 

A kev goal in the design of aircraft cockpits, aircraft operating procedures and 
crew draining is the reduction of incidents and accidents attributed to human 
error Some have claimed that automation can eliminate human error by 
removing the pilot from the control loop. Others have claimed that while some 
types of error may be reduced the automatic equipment itself introduces 
opportunities for new types of human error. The new equipment may eliminate 
small errors but introduce the possibility of large errors. These new error forms 
seem to be particularly difficult to anticipate during the design phase. 

The group discussed: the changes in cockpit systems that have affected the type 
and frequency of crew errors; examples of types of human error that have been 
reduced; and examples of new types of human error. 

The key output of this working group was a list of “high” priority and “medium 
priority automation issues and recommendations that relate to errors and error 
detection in current and future advanced technology cockpits. 
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HIGH PRIORITY ISSUES 
Industry Wide Error Data Rase 


The design of aircraft cockpits is an evolutionary process. Each new cockpit 
design is an attempt to improve on the past designs. If the cockpit designers know 
that pilots systematically make specific errors in operating a piece of equipment, 
they can often design the new equipment so that an error is either impossible or 
much less likely. To make this process work the designers must know about the 
types of error that are occurring. Currently there is a large body of operational 
experience which is not known to the flight deck designers. An industry wide data 
base should be established to record errors and incidents that can be used for 
design of future systems, upgrades to current avionics software or changes in 
current training courses and procedures. The IATA has an incident data base that 
might be adapted to this use. 

Training for Hi ghly Automated Aircraft 

Training should be organized so that the pilot can always answer the following 
questions about an automatic system: What is it doing? Why is it doing it? and 
What will it do next? The pilot needs to know the system well enough to be able 
to predict what it will do in different contexts. Training should contain 
information on how the designers intended the system to operate (e.g. FMS and 
autoflight). This information is often lost in the long chain between designer and 
operator. 


Error Detection & Co rrection: Self and Automatic 

Humans make and usually detect errors routinely. The same mental processes that 
allow humans to cope with novel problems can also lead to error. Bill Rouse has 
argued that errors are not inherently bad but their consequences may be. He 
proposes the development of “error-tolerant” systems that detect errors and take 
steps to prevent the consequences of the error from occurring. Research should 
be done on self and automatic detection of random and unanticipated errors. For 
self detection, displays should be developed that make the consequences of errors 
immediately apparent. For example, electronic map displays graphically show the 
consequences of horizontal flight plan entry errors. Vertical profile displays 
should be developed to make apparent vertical flight planning errors. Other 
concepts such as energy circles could also help the crew detect gross flight 
planning errors. For automatic detection, systems should be developed that can 
track pilot activity, infer pilot intent and inform the crew of potential errors 
before their consequences are realized. Systems that perform a reasonableness 
check on flight plan modifications by checking route length and magnitude of 
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course changes are simple examples. Another example would be a system that 
checked the aircraft’s planned altitude against a data base of world terrain 

elevations. 


Simation/Svstems/Configurati on Awareness 

Comprehensive knowledge of current status is necessary to make appropriate 
error-free decisions. In autoflight control systems, the pilot should know how 
close the system is to its performance limits. Trend information should trigger 
annunciations of potential loss of control authority problems. For example the 
message, “You are using up your control authority,” might have been helpful to 
the crew of the China Air flight that lost control of a 747 on a flight to San 
Francisco after a single engine failed. Similarly after a subsystem failure, the 
pilot should be able to call up displays showing the consequences of the failure in 
terms of remaining subsystem functionality and any new operational limitations. 

Derision Support and Information M anagement 

Systems should provide information appropriate for the current flight situation. 
This could include suppression of non critical information during critical phases 
of flight. The system should also be capable of answering WHAT, IF and WHEN 
questions to support the pilot in exploring options and deciding on a course of 

action. 


MEDIUM PRIORITY ISSUES 

Cockpit Resource Management (CRM) and 
Crew Coordination in Advanced Technolo gy Aircraft 

Good cockpit resource management (CRM) is an important element in the 
detection of crew errors. The CRM concept was developed with older lower 
technology and may need to be updated for the new two-crew advanced 
technology cockpits. 

Standardization 

Standardization of equipment would reduce errors due to transitioning between 
cockpits but it may have a negative impact on progress. We do not want to 
standardize on an error-prone and difficult to use design. Unfortunately, 
standardization is most important on complex systems like the FMS that most 
need to be improved. In addition, renewed emphasis should be placed on 
standardization of fundamentals which affect human performance. 
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The Minimum Equipment List 


Operational decisions as to what equipment is on the minimum equipment list 
(MEL) should consider the human role. It was felt that the designer’s concept of 
how the aircraft should be operated was sometimes compromised by decisions 
which allow the aircraft to operate when certain equipment is not functioning. 
Flying with MEL items should not result in operation below a minimum level of 
capability. This minimum level should include the normal displays. It was felt 
that much valuable training time could be saved if training for operations in very 
rare backup configurations was not required. 

Error Management: Inform vs. Protect 

Should an automatic system inform the crew that they are about to exceed the 
performance envelope of the aircraft or should it unilaterally prevent the aircraft 
from exceeding its performance envelope? This issue is closely related to the 
larger issue of the proper role of automation in the cockpit. It is a very important 
issue but there may not be any empirical way to address the problem. It is also 
not an “all or none” issue. An “inform” cocoon could be inside of the “protect” 
cocoon. A protect cocoon could be turned off in certain situations at the pilot’s 
discretion. 


On-Line Help 

Help functions should be provided on the control display unit (CDU) for 
nonroutine operations to reduce the need for in-flight consultation of the manual. 
This will help insure the optimum use of equipment and help prevent errors. 

Workload Management (Boredom and High Workload 

Resolution of the role of automation in the cockpit should enhance workload 
management. In addition, during long flight legs, consideration should be given 
to on line training of complex systems such as the flight management system 
(FMS). Other possibilities are interactive electronic flight manuals. 

Error-Focused Design Methods 

Theories of human error and design guidelines developed by cognitive scientists 
should be applied to the design of new cockpit systems. For example, Professor 
Donald Norman’s new book, “The Psychology of Everyday Things,” describes a 
theory of human action and error and offers numerous guidelines on how to 
design systems that are easy to use and less error-prone. 
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Human Factored Certification Methods 

The certification process should include explicit human factors criteria. 
Numerous methods have been developed in the field of human-computer 
interaction for evaluating the adequacy of an interface design. These methods 
should be adapted to provide more objective evaluation methods and guidelines 
for human factors certification criteria. 

SUMMARY 

In addition to these specific automation issues and recommendations, the group 
members felt that defining the proper role for the automation in a human- 
centered aviation system was of fundamental importance in the prevention and 
detection of crew errors. 
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INTRODUCTION 

The issues of design and certification of transport category aircraft are both 
complex and interrelated. Certification regulations, to some extent, do effect 
design decisions. But the current certification criteria, relating primarily to 
workload and automation factors, are not specifically identified. The working 
group on design philosophies and certification was charged with identifying the 
major factors underlying the effective use of automation technology, its 
introduction into an operational environment and directions for the future in 
design and certification. 

A major question which must first be addressed is, “Why should automation be 
introduced onto the flight deck?” Although many answers could be given to this 
question, among the most important are! 

1) It allows the flight system (crew + aircraft) to attain a broader 
operational capability. 

2) Economic efficiencies can be more easily obtained. The most 
notable example is in fuel management. 

3) System consistency is improved. The introduction of automation 
allows for the reduction of operational variability. 

4) Automation has clearly enhanced safety. 
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With these positive aspects of automation identified, it is now possible to proceed 
with a discussion of how the design and certification of automated systems might 
be improved and what are the major issues surrounding such an improvement. 

DESIGN 

The issues surrounding the design of automation for the flight deck are complex 
and interrelated. The discussion of this topic is organized into the following six 
subjects: the design of the interface, the pilot’s role, evaluation criteria, current 
performance, impact on the NAS as a whole, and redesign/retrofit issues. 

Design of the I nterface to the Automated System 

Among the most central of the issues is the design of the interface to the 
automation since the crew must communicate with the automation through this 
interface. This communication will be required regardless of the ultimate role or 
responsibility assigned to the crew. “Unfriendly” interface designs seem to be 
among the most common complaints about automatic systems. This is particularly 
true in the design of those automatic systems which are affected by the ATC 
environment. Unfriendly interfaces are a bigger problem when there is a 
requirement to modify the data/instructions to the automation under the pressure 
of time and/or short notice. 

Interface designs must also assure appropriate situation awareness. Careful 
attention must be given to the effects of consistent presentation of status 
information and prioritization of cautions and warnings. Preservation, where 
possible, of tactile cues to maintain awareness is also important. 


For computer controlled systems, consistency in data entry, information 
retrieval, and procedural aspects are very important. Page seeking should be 
minimized and on-line “help” should be available as necessaiy. Systems such as 
the PMS, FMS, INS, OMEGA, ACARS, TCAS, and Mode S were among the 
systems felt to require the most careful interface construction. 


Automation, particularly that associated with the CRT or glass cockpit, may also 
be a distraction in the cockpit. The extent, or even whether this is a serious issue, 
is not known. The issue of concern is the time and/or attention dedicated to 
adjusting or changing the automatics as opposed to flying the aircraft. 


Role of the Crew and Automation 


A critical question both in this working group and throughout the conference are 
the roles of the automation and the crew. The preponderance of opinion was that 
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the automation should take a greater role in the basic stability and control of the 
aircraft, particularly for aircraft which have relaxed and/or unstable flight modes 
(e.g. tilt rotor concepts). Higher level functions, however, such as flight 
planning/replanning, system status management and decision making, should not 
be completely relegated to the automation. 

This view is primarily based on the fact that the fundamental pilot functions have 
not changed and are not expected to change regardless of future improvements to 
aircraft and/or the NAS system. In particular, the basic flight crew functions are 
to: 


aviate, navigate, communicate, and operate. 

The goal of automation should be to aid the crew as they participate in the task 
and manage the aircraft as a system. 

With respect to operating philosophies, there is a difference between automated 
and non-automated aircraft. The responsibility of the pilot is the same in both 
types of aircraft but the actual activities required to accomplish these tasks are 
very different depending on the level of automation. In recognition of this fact, 
several carriers have elected to divide their fleet so that pilots do not fly both 
automated and more manual aircraft in the same time frame. 

One central issue regarding the role of the pilot and automation is that of designs 
which try to provide error protection. Envelope protection has been cited as an 
example of this form of error proofing. On the positive side of this issue, this 
trend is consistent with the historical trend toward increasing automated 
intervention, particularly in the guidance and control area. On the negative side, 
however, such error proofing is not as reliable as hoped. These systems have 
failed and will continue to fail. Although the design goals hoped to achieve failure 
rates of less than 10 to the minus 9, the actual operational experience has been 
(and will continue to be) much lower than this number, because designers are also 
human. They are also subject to error and, in particular, are not able to anticipate 
every possible operational situation, particularly in the complex aviation 
environment. 

It is, therefore, very important to provide a manual override for automated 
systems and to provide the pilot with the mechanisms for a greater role in 
decision making. In particular, it is important to distinguish between those 
automated systems which completely perform the task and those which aid the 
pilot so that the crew is a participant in the system. 
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Another question is whether a high level of automation would violate FAR 91. 
Automation should not take away the Pilot-In-Command’ s authority and 
responsibility. In short, the design should not isolate the human from the ability 
to intervene and manage the aircraft at all times. 

The role of the crew is summarized in Figure 1. The central circle, situation 
dominance, indicates that the flight crew must maintain “legal” awareness and 
control/dominance over the aircraft status. The crew must have the information 
and ability to exercise operational judgement, contingency management, systems 
management, and flight planning and replanning. In addition, all of these 
functions must have an appropriate “manual” capability. The role of the 
automation should be to support the crew particularly in guidance and control, 
navigation, and system monitoring. 


Evaluation Criteria for Automated Systems 


Several characteristics were suggested which could be used to define an 
appropriate implementation of automation versus a poor implementation. As 
discussed above, it is very important that the implementation be user friendly. 
Ease of training is a natural corollary to such an implementation, since a good, 
self explanatory design can reduce die need for extensive training. In addition, 
functional adaptability is important. More definitive guidelines or criteria need to 
be established so that designers can evaluate systems as early in the design phase 
as possible. 


Performance of Current Automated Systems 

The general discussions regarding current automated systems indicated that 
functional systems such as the FMC (flight management computer) are the ones 
with the principal issues. Other automated systems such as those for fuel and 
pressurization work well. Again, the central issue seems to be that of designing 
interfaces which can facilitate and support the active role of the crew. 

More specific information could also be used in this area. To what degree, if any, 
are the operational crews uncomfortable with the kind or degree of automation? 

IMPACT OF AUTOMATED FLIGHT DECKS 
ON OTHER PARTS OF THE NAS 

The National Air System (NAS) is composed of many interacting elements. A 
new technology implementation in one part will necessarily impact the other 
elements. The introduction of new automated technology onto the flight deck has, 
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however, been incremental. This is, of course, the usual method for introducing 
new technology. It can, however, cause problems to occur particularly where the 
older technology must interface with the new. 



Figure 1 


For example, the ATC system is currently in the process of being updated, but 
automated flight decks must currently interface with the existing, less-automated 
ATC technology. Although there is clearly an impact from the incremental 
introduction of automation, the magnitude is not understood. Future 
implementations would do well to study the impact of a new implementation on 
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the total system, before it is introduced so that potential system-wide anomalies 
could be evaluated before the operational phase begins. 

Redesign/Retrofit Issues 

The current organizational process for designing and approving systems does not 
adequately address the issues associated with redesign and retrofit. In the 
introduction of a new technology, lessons are frequently learned or apparent only 
during the operational phase. This may necessitate a redesign and/or a retrofit 
into the existing fleet. However, the cost of redesign, certification, and retrofit 
often make the actual implementation impractical and operations may continue 
for years using the existing system supplemented only by aircrew training. The 
panel report on operational experiences cited some examples where this type of 
operational “crutch” had not been effective. 

Summary of Design Directions for the Future 

The major issues associated with the design of future automated aircraft are 
summarized below. 

1) ROLE OF THE FLIGHT CREW: SITUATION DOMINANCE 

- Pilot’s Role Must not be Compromised 

- Tasks can be Appropriately Delegated to Automation 

2) ROLE OF AUTOMATION: AIDING THE FLIGHT CREW 

- Optimize Resource Efficiency & Economy 

• permit crew to do more in a complex environment 

• achieve better perspective of “big picture” 

Reduce Operational Variability 

Enhance Safe Operations 

3) DESIGN PHILOSOPHY: HUMAN-CENTERED AUTOMATION 

- Achieve Objectives Cited Above 

- User Friendly 

Adaptable to Function/Option 

- Support Crew in Functions which do not Compromise Crew’s 
Role of Situation Dominance 
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4) INTERACTION OBJECTIVE: CREW INVOLVEMENT & 
PROFICIENCY 

- Use a Systems Approach to Assure Flight Crew Manual & Cognitive 
Proficiency 

Designs with Crew as Backup should Permit Ease of Task Recognition 
& Performance without Confusion, Hesitancy, or Lack of Skill 

- Eliminate Growing Tendency to Peripheralize Crew Involvement and 
Function 

5) CRITICAL DESIGN NEED: DETERMINATION OF BENE- 
FITS AND LIMITATIONS OF STANDARDIZATION 

CERTIFICATION ISSUES 

The certification issues are broadly grouped into 3 categories: Certification 
policy, validation of automated systems/software, and human factors criteria. 

Certification Policy Issues 

The current FAA philosophy regarding certification generally does not attempt to 
influence the introduction of new technology in any way. The position is largely 
one of allowing industry to move as necessary to build the best possible aircraft. 
The FAA role is simply one of evaluating the “safety” of the resulting aircraft. 
As we have seen, however, the introduction of new technology itself can 
contribute to operational safety issues. This is particularly true due to the rapid 
introduction of advanced, computer based technology onto the flight deck. 

The philosophical issues surrounding the question of constraining the introduction 
of new technology are complex. No specific recommendations regarding this 
issue were made by the group, but it is a factor which should receive further 
discussion. 

Another philosophical issue involves certificating to standards versus certificating 
to guidelines. Most other aircraft systems such as structures, are primarily 
hardware based and specific numerical standards can be developed. Automation, 
however, is a combination of many disciplines including computer hardware, 
software and human factors. The complex interaction of the components does not 
always lend itself to specific standards. In addition, the technology is changing so 
rapidly that it would be difficult to develop exacting standards which could be 
effective over the next decade. In view of these factors, the use of guidelines for 
automation as opposed to standards should be discussed. 
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A last policy issue, but one of primary importance, is the way that the 
certification process considers (or does not consider) the trade-off between 
understandability and training. Since the design of the interface to an automated 
system was an important issue in the previous section, the philosophy of 
certificating to the “understandability” of an interface is most important. 
Currently such trade-offs between the need for training and the complexity of the 
design are not usually apparent. Perhaps they should be. 

V alid a tion ol Au tom a te d Syst ems/Spftwa r e 

Software validation is not usually an easy process and yet it is very important to 
the proper functioning of most automated systems. The certification process must 
insure that the software will function as expected under all conditions. Yet, is it 
possible for the certification process to adequately make this assurance? How can 
this best be accomplished? 

Human Factors Evaluation Criteria 

The hardware components of an aircraft receive substantial testing before they 
are certificated. This testing typically involves stressing the structure to the 
failure point in a “shake and bake’ manner. The automated systems aspects, 
particularly those associated with human factors, do not receive such thorough 
testing. This is partially true because such tests and procedures are only now 
becoming available. An issue of concern is the extent to which more and better 
human factors testing of systems is required prior to certification. 

Summary of Future Directions for Certification 

The major issues associated with the future directions for certification of 
automated aircraft are summarized below. 

1) RE-EVALUATION/DESIGN REVIEW REQUIREMENT 

- No Certification Standards can be Appropriate over the Entire 
Life of an Aircraft 

- After a Reasonable Service Time, Operational Shortcomings 
Should be Addressed 

2) FULL/PART MISSION SIMULATION TO TEST DESIGN 

- To Help Prevent Costly Redesign 

- To Permit Certification Staff to Observe and Prepare for the 
Flight Test 
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3) REVISE AND STRENGTHEN CERTIFICATION PROCESS 

Crew Interaction 
Automation Interface 

- Human Performance 

4) ESTABLISH INSTITUTIONALIZED COMPREHENSIVE FEED- 
BACK SYSTEM 

- To Identify Opportunities for Subsequent Design Review 

- To Anticipate Next Generation of Aircraft 


5) ESTABLISH MULTI-DISCIPLINARY CERTIFICATION DESIGN 
REVIEW 

- At Beginning of New Aircraft Development to Learn, Review, 

Critique, etc. 

6) ESTABLISH CRITERIA FOR ASSESSMENT OF HUMAN 
PERFORMANCE DESIGN AND ENGINEERING 


Provide Design & Engineering Guidelines 
- Assist Certification & Review Processes 


7) FLIGHT DECK AUTHORITY FOR FAA CERTIFICATION 
AND FLIGHT TEST PERSONNEL 

8) ESTABLISH BETTER COORDINATION AND COMMUNICATION 
WITHIN FAA RE: AIRCRAFT DESIGN/CERTIFICATION 
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SUMMARY and CONCLUSIONS 

Susan D. Norman 
Chair 


INTRODUCTION 

The material presented at this workshop provided a particularly comprehensive 
and broad overview of automation in the air transport system today. It is, 
therefore, not easy to select the most important points from the workshop and 
prepare the conclusions. This section will focus on the major, global themes. 


SUMMARY 

A very condensed summary of the major ideas and concepts presented in the 
panels and papers is given here in order to form a basis for the conclusions. A 
brief section on common themes of the Working Groups is also included. 

Design Issues 

Although specific data regarding accident/incident rates for automated versus 
non-automated aircraft are not readily available, it is clear that automated aircraft 
have a good, operational record. For example, one carrier stated that they have 
been flying the Boeing 767, one of the first aircraft to employ substantial 
automation, since 1982 and they have never had an accident or incident resulting 
in damage to the aircraft. 

One manufacturer cited principles for effective system design in the following 
priority: simplicity, redundancy, automation. The goal is to use automation when 
necessary, but automation should not be a goal in itself. Design issues are first 
solved with simplicity, then redundancy, and finally automation. 

Certification Issues 

A fundamental issue with respect to certification is the lack of comprehensive 
human factors requirements in the current rules. Although the reasons for this 
situation are complex, the result is that design engineers must necessarily make 
critical design decisions long before the flight tests occur. In the absence of rules 
which incorporate human factors criteria, the question is whether or not the 
concerns of the FAA test pilots and the line pilots he represents are adequately 
considered. 
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With respect to certifying automated aircraft in today’s environment, several 
factors were cited as important. These are: 

• Crew alerting systems 

• Manual operation of an automated aircraft 

• Crew over-confidence in automation 

• Identifying circumstances where automatic protection is a clear 
benefit (i. e. angle of attack protection, etc.) 


Operational Considerations 

Although automation has been a clear benefit, some factors were cited which have 
been involved in incidents with automation. These include: 

• Inadequate operational knowledge on the part of the crew 

• Inadequate cockpit discipline and allocation of responsibilities 
between the pilot-not-flying and the pilot-flying 

• Inadequate cockpit resource management 

• Operation in split mode (e. g., operation of autopilot without 

autothrottle) 

• Poor switch discipline 

Lessons Learned from Non-Aviation Automation Experiences 

Numerous examples have been cited throughout this document of lessons learned 
from our current understanding of designing and operating automated systems. 
Dr. Woods summarized these as follows: 

• Shifts in automation have changed the human role in unforeseen 

ways. 

• Critical human role is to adapt to unanticipated situations. 

• Automation changes required human skills, but does not eliminate 

them. 

• Automation introduces new error forms and types of system 

breakdown. 

In designing for human centered automation, the machine/automated system 
should always provide support for the human. It should display information on 
what it is doing, what it will do next and why. Provision of support for the 
human role in error detection and recovery is also most important with 
computerized systems. 
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Field Studies in Aviation 


Dr. Wiener reported his interim findings of a study of two air carriers and flight 
crew evaluation of the B-757. 

• High enthusiasm for the B-757 

• Training is good, overall, but there is too much emphasis on 
automation rather than the basics. 

• ATC limits the use of some features ( e. g., VNAV) 

• Workload may be increased or decreased. 

Captain Peter Heldt, chief technical pilot for Lufthansa, reported the results of a 
pilot questionnaire on the A3 10-200. Overall, the pilots liked the automation, but 
“keeping the pilot in the loop” was cited as mandatory for automated systems. 

Advanced Automation Sys tem (AAS) for ATC 

The schedule for implementation of the AAS calls for the first site delivery at the 
Seattle Center in April, 1992, with the equipment expected to be operational at all 
sites by June, 1995. The system will provide: 

• New automatic separation-assurance techniques 

• More direct, conflict-free routing 

• Better traffic metering techniques 

• Increased controller productivity 

• Capacity to handle projected air traffic growth 

• Tie together all FAA primary enroute and terminal ATC facilities 

• Provide greater system reliability (max. down time - about 2.5 
min/yr) 

Common Themes in the Work ing Group Reports 
Several themes were repeated in the working group reports. Some of these are: 

• Design of the pilot/controller interface is crucial and it must provide support 
such as status annunciation and display of an appropriate level of situational 
awareness for their intended roles. 

• The air-ground interface design is critical and yet there is an apparent lack of 
coordination/research in this area. 

• Definitive human factors criteria need to be developed. 
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• Since aircraft automated systems must support the role of the pilot, it is most 
important to understand this role and to develop some level of consistency for 
this role among the various components of the aviation industry from 
airframe manufacturers to operators. 

CONCLUSIONS 

Automation is a Clear Benefit 

Although most of the text of this report relates to issues regarding improvements 
to our design, operational use of and training for automation, an important fact to 
remember is that automation has substantially improved the operational safety 
and efficiency of our air transport system. As with the introduction of any new 
technology, there are components and factors which cannot be clearly and 
completely understood without the severe and challenging test of day to day 
operations in a real environment. 

Aviation is perhaps one of the most demanding, real-time environments for use 
of any new technology due to the vast number of daily operations and the 
complexity of the ATC interaction, weather, etc. The safety record for automated 
aircraft, however, is very good. The relatively smooth introduction of cockpit 
automation into the existing system must be taken as a tribute to the technical 
ability of the aviation community. 

Even though the safety record for automation appears to be good compared to the 
previous generation of aircraft, the potential for improvement in our ability to 
design, certificate, use and train for automation was apparent at the workshop. In 
particular, the aviation environment is not one where flights always proceed as 
planned. Weather problems, ATC clearance revisions, emergencies and 
equipment malfunctions, and even combinations of these events, occur more 
frequently than we would like. 

Yet automation is a technology which works best in predetermined situations such 
as those which can be planned and programmed ahead, either at the time of 
design or on the ground before take-off. Automated systems, because they are 
really machines, are designed or programmed primarily to handle the “normal” 
situations which account for most of the flying hours. However, the technology is 
not yet well enough developed to provide quick and easy flexibility when the 
external situation changes. 

This phenomenon has been called brittleness” and it is a characteristic which has 
been associated with automation (particularly expert systems). It is not wrong in 
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itself; it is merely a limitation of the technology which must be considered in the 
design, certification, training and procedural use of these systems. Non-aviation 
industries have experienced the same phenomenon and much can be learned from 
their experiences (refer to the Woods paper in this document). 

As a result of factors such as brittleness, irregular operations in an automated 
environment can become the most troublesome. Irregular operations, discussed in 
more detail in the working group report on crew performance, are flights in 
which normal operations have been interrupted or are no longer possible. In 
these situations, the pilot’s operational understanding of the system, and how it 
will perform under the varying conditions, becomes crucial. The design must 
support this pilot role by providing appropriate display information. In addition 
under these circumstances, the human role is often to maintain the boundaries of 
the automated system, perhaps by methods such as inserting an erroneous wind 
vector, so that it can function effectively. 

Understanding (Figure 1) thus becomes a key concept. The ability of the flight 
crew to understand the automatics must be supported by all phases of the aviation 
process from design through training and operations. 


UNIDEIRSTANMNG Is a 3£®y (C®m««|ptl 

• Of the Way it Works 

• Of the System Intent 

• Of Control Laws 

• Of Normal Versus Irregular Operations 

• Of the Implications of System Status 


Figure 1 

A point must be made regarding the actual systems which have been automated. It 
was generally agreed that the automation of aircraft subsystems has been much 
less troublesome than other systems such as those which support navigation. For 
example, autofuel and autothrottle systems have functioned very well and their 
operation appears to be easily understood in the cockpit. These systems, however, 
do not depend on external information sources such as pilot data entry and 
ground transmitting stations. As such, they do not always have the inherent 
interface complexity of other automated systems, such as those used for 
navigation. 
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Because the factors which affect the performance of an automated system can be 
complex and may not always be immediately apparent, the crew communication 
in the cockpit and with ATC about the operation of the aircraft must compensate 
for this situation. In addition, actions on the part of the automatics may 
sometimes be transparent to the crew. In older technology aircraft, these actions 
are performed and checked by a crew member. In summary, the use of 
automatics necessitates closer crew communication as well as a closer interaction 
in all phases from design to operations (Figure 2). 


CLOUE UNTBlRACTniONS 

A Close Interaction is Required in All Elements of: 

• Design 

• Training 

• Operations 

• ATC 

Examples: 

• Between the Automatics and the Flight Crew 

• Between Crewmembers on the Flight Deck 

• Between Flight Deck Design and Operational Procedures 

• At the System Design Level (System Level Perspective) 

• Between the Flight Deck Crew and ATC 


Figure 2 


Comments from the Participants 

The following comments from verbal remarks and anonymous evaluation forms 
may help to provide a basis for understanding the essence of this document. 
Participants were asked the most important new idea or issue learned at the 
conference: 
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"The basic human factors associated with automation are generic and 
are UQL being adequately addressed by either ATC or cockpit 
automation designers.” 

“Man must not be replaced by but rather learn to make 
effective/ efficient use of automation in the aircraft environment” 

“The problems generated by introducing automation in the cockpit 
are very similar to the problems encountered in automating the 
control tower...” 

“Automation is working very well in many applications and users 
want more, not less; but there are problems that need work.” 

“The automation problem is very broad and it is very important to 
attack it systematically.” 

A valuable part of the conference was the exchange of ideas on 
... “how others are coping (training and operational tips), what design 
changes and enhancements are in store and the concepts of automatic, 
semi-automatic, and manual operation when control is necessary in a 
time constrained or irregular operation.” 

Direction for the future is also an important issue and several pertinent comments 
were made regarding it. 

“The role of the pilot should not change. Automation has to be added 
carefully. There needs to be early design guidance to accomplish 
good automation.” 

“We must find ways to introduce the equipment more effectively. In 
my training experience, a number of pilots were very reluctant when 
first introduced to the new equipment." 

One of the most critical areas to emerge as a need for the future is 
the “ development of improved interaction between the air and 
ground sides of the National Air System.... / am concerned that the 
cockpit is expanding beyond the ATC capability. This may cause the 
airborne side to be incompatible with the ground operations and 
pilots may not use new systems.” 
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The industry is trying to run with its brakes on. Information dissemination, 
particularly to the higher levels of manufacturing and operational management, is 
essential. 


The Role of the Pilot 

One important issue to emerge from the conference concerns whether or not the 
role of the pilot has changed as a result of automation. Several controversial ideas 
were discussed and I have elected to quote from several letters written to me as 
chair after the close of the conference. Drs. Rolf Braune of Boeing and Earl 
Wiener of the University of Miami most articulately presented the relevant issues 
in this debate and their views are cited here. 

The definition of terms becomes crucial in this discussion and “goal,” “role” and 
“functional level” are all important aspects. Regarding the goal of the pilot/ flight 
crew. Dr. Wiener writes that the goal of the pilot is: & 

....to fly the aircraft from A to B with maximum safety, minimum 
cost, and maximum passenger comfort. This strategic goal is 
unchanged, but in order to achieve this goal, tasks and subtasks are 
carried out, and these, of course, have been altered by automation 
and other cockpit equipment. .." 

As stated, this goal (to fly the aircraft from A to B ) for the flight crew is 

relatively uncontroversial; however, the issues of role and the impact on the 
specific tasks which the flight crew must now perform in an automated 
environment are more difficult to assess. 

Regarding the role of the pilot, Dr. Rolf Braune pointed out that: 

...the role of the pilot has not changed and probably will not change 
until Federal Air Regulation (FAR) 91 .3 is changed. This regulation 
states that the pilot in command of an aircraft is directly responsible 
for, and is the final authority as to, the operation of that aircraft. 

This is a clear definition of a role. As part of this role, the pilot 
always performed the functions of monitoring and managing. What 
the automation technologies are attempting to do is to provide the 
pilot with the tools which will simplify necessary tasks or eliminate 
unnecessary tasks." 
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Dr. Wiener, however, concludes that the role of the pilot has changed. He writes: 

u Role refers to a more global construct than a list of tasks. It refers 
to the function of the crew member in realizing his strategic goal. 

This function, or role, has been altered by the tasks. The pilot s role 
is more passive, less manual, more cognitive, etc. To fail to 
recognize this is a serious impediment to our understanding and 
eventual resolution of the ' automation problem . 

The role of the pilot is changing and our job is to understand the 
implications of this change.” 

Although the above basic statements of the issues are equally valid, both points of 
view draw very different conclusions regarding the apparent change in the role 
of the pilot. Clearly, FAR 91.3 gives the pilot in command ultimate authority and 
in this sense, the pilot’s role has not changed. But the methods which pilots use to 
perform their duties have been dramatically altered in the automated 

environment. 

Dr. Braune further clarifies the important difference here when he writes, 

“The apparent disagreement over whether the role of the pilot has 
changed or not may arise because we are looking at different 
functional levels of the pilot's task hierarchy. Afunctional hierarchy 
of the commercial pilot's task environment should be developed. 

This would help to identify: 

• those functions which may have changed, 

• effects of the change, 

• ways to counter those with negative effects, 

• level of these functions. 

In addition, there is a limited amount of objective data to help focus 
on the real issues. We may be confronted with over-generalizations 
which do not help understand the real causes of the issues we have 
discussed, particularly in light of the highly favorable accident 
statistics for advanced technology aircraft.” 

In summary, the consensus at the conference was that the role of the pilot should 
not change. The flight deck crew must be actively involved in the operation of the 
aircraft and the substance of FAR 91.3 should remain in effect. However, as the 
tools and methods used to fly aircraft change, the choices which a pilot can 
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realistically exercise may also change. It is in this sense that a fundamental change 
in the pilot’s ability to accomplish the intended role may occur. 

Unfortunately, as Dr. Braune has correctly pointed out, there is a paucity of data 
regarding the actual, operational task responsibilities of the flight crew. Due to 
this lack of data, only opinions can be given. As chair, it is my opinion that the 
intended role of the pilot has not changed, but that the actual ability of the flight 
crew to exercise much of the assigned role/authority has indeed been changed by 
many factors including the ATC environment as well as the implementation of 
current flight deck automation. 

Dr. Braune indicated the need to understand the pilot’s task hierarchy more 
clearly and Dr. Wiener suggests that it is our job to understand the magnitude, as 
well as the implications, of these changes. Both of these needs are important 
topics for future research, particularly if we are to understand clearly how to use 
automation technology more effectively. 

DIRECTIONS FOR THE FUTURE 

Although much has been accomplished regarding automation of the flight deck, 
there is still much to do. The workshop provided a valuable opportunity to 
exchange experiences regarding operations and training for automated aircraft. 
Continuation of such opportunities needs to be provided by the relevant 
organizations and government agencies. 

The need for quantifiable data regarding the implementation and interface designs 
for automated systems is also very apparent. Full mission simulations which 
compare alternate displays, etc. and test the performance of the entire system 
(ATC + flight crew + aircraft) would be very helpful. 

Improvement in the human factors aspects of the certification process is needed. 

Additional research is needed to better understand how to support the human role 
in an automated environment. This involves interface design, training and 
operational procedures. 

Improving our understanding of the air-ground interface is crucial for the future. 
This development is necessary if coordination of the planning and implementation 
of advanced automation for both the flight deck and the air traffic controller are 
to proceed in a integrated manner. Automation provides the potential for one part 
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of an automated system to impact another, sometimes without direct human 
intervention. As a result, it is important to examine the implementation of 
automation as an overall system so that the design implications can be made 
visible before the operational phase commences. 


159 



APPENDICES 


p»<fcCEDU*Ca E/wo!:. 


E ELAN51 riOT 


1^0 iHTtHHOUW.^ 




APPENDIX A 


AIRCRAFT AUTOMATION PHILOSOPHY: 
A SOURCE DOCUMENT 


This report was prepared to provide 
an initial basis for discussion for the participants in 
the NASA/Industry/FAA workshop, 

“Flight Deck Automation: Promises and Realities,” 
Carmel Valley, California, August 1-4, 1988. 


NASA Ames Research Center 
July 1988 
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This report was prepared based 
on contributions by the following participants 
at a preliminary workshop on 
Philosophy of Flight Deck Automation 
held at Carmel Valley, 

April 25-26, 1988 


Susan Norman 
Charles E. Billings 
David Nagel 
Everett Palmer 
Earl L. Wiener 
David Woods 

with contributions by: 
Ren Curry 
Norm Geddes 
Grace Pariante 


Automation: “automatic operation or control of a 
process, equipment, or a system” (American Heritage 
Dictionary, 1976) 


Automatic: “acting or operating in a manner essen- 
tially independent of external control; self-regulating” — 
but also, “lacking volition, intention, or conscious plan- 
ning” (American Heritage Dictionary, 1976) 
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AIRCRAFT AUTOMATION PHILOSOPHY: 
A SOURCE DOCUMENT 


1.0 Introduction 

Assessing the impact of automation on civil transport aviation is difficult and 
complex. Although many major benefits have been realized from the introduction 
of new technology onto the flight deck, there is an underlying concern about its 
implications and long term effects. 

One recent report which reflects broad aviation concerns states, “The recent and 
rapid introduction of advanced computer-based technology onto the flight deck of 
transport category aircraft has been associated with a dramatic change in both 
aircraft operations and the role and expertise expected of the flight crew. 
Although no specific accident statistics are available, there have been a number of 
serious incidents, which necessitate development and testing of a critical, 
scientifically based philosophy of automation” (Norman, et al., 1988 p. 1). 

A discussion of the framework for a “Philosophy of Automation” and the need 
for its development form the basis for this paper. An automation philosophy can 
provide guidelines and constraints for answering design questions and a 
methodology to evaluate both individual design decision and the overall utility of 
the automation. 

Beginning with a historical context for issues related to automation, this paper 
explores the consequences of current automation and describes a direction for the 
future which could lead toward a Human-Centered Philosophy of Automation. 

2. The Evolution of Automation in Civil Transport Aircraft* 

This section describes the historical evolution of automation technology, in the 
hope of discerning trends in the respective roles and functions of automation and 
of the humans who will operate these aircraft. 


* This section is reprinted from an unpublished manuscript entitled, “Toward Human Centered 
Automation”, with the permission of the author. Dr. Charles E. Billings. 
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2.1 The Beginnings 


In the earliest days of manned flight, automation technology was needed to 
stabilize the aircraft attitude by directly controlling the aerodynamic surfaces. 
Gyroscopic devices were well suited to this task. 


In 1891, Sir Hiram Maxim secured a British patent for a gyroscopic stabilizer. Five 
years later, it was installed and tested on a steam-powered flying machine (Pallett, 

1983). Orville Wright was more successful with an automatic stabilizer that 
activated the wing- warping and elevator mechanisms associated with control of roll 
and pitch. The device was patented in 1913 and later won a Collier Trophy for its 
inventor (Prendergast, 1980). 

The following year, Lawrence Sperry received a French safety award after the 
successful flight demonstration, in Paris, of a two-axis system (Heinmuller, 1944). 

In 1918, H. J. Taplin patented a system that relied on aerodynamic pressures. His 
device was successfully flown in 1926 in the United States, but is not known to 
have been used thereafter (Taplin, 1969). 

2.2 Early Autopilots 

Gyroscopic devices have dominated aircraft inner loop control (maintenance of 
attitude for all spatial axes) since that time. 

A Speny “automatic pilot” was installed in the Winnie Mae in 1932 and was used 
extensively by Wiley Post in his successful circumnavigation of the globe in 1933. 

A later model Sperry gyro pilot was installed in the Lockheed Electra which 
Howard Hughes used for his global flight in 1939. 

By the beginning of World War II, autopilots were installed in a variety of long- 
range aircraft. With vacuum-driven gyroscopes (which often provided cockpit 
attitude and heading reference as well), these devices were the cornerstone of 
automatic flight throughout the war and for several years thereafter. They 
alleviated fatigue and freed pilots from the burden of constant manual aircraft 
control. 

Following the war, autopilot technology advanced rapidly. Vacuum-driven gyros 
were replaced by more effective electrical systems and processing of autopilot 
error signals was taken over by electronic amplification. 

The transition of radio navigation from low frequency ranges and non-directional 
beacons to very high frequency omnirange (VOR) transmitters simplified the 
navigation process, insulated it from electrical storm interference, and provided 
more precise directional data. The output of these radio aids, when coupled to 
autopilots, provided the ability to track radials with respect to a VOR station. The 
output of the VHF instrument landing system (ILS) was also used to provide 
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precise tracking of localizer beams, while the corresponding glide slope 
transmitters were coupled to pitch axis control for vertical guidance. 

Precise data regarding magnetic heading, altitude and position with respect to 
external references, when integrated into the autopilot system, enabled 
considerable outer loop, as well as inner loop, control. When commercial jet 
aircraft were introduced in the late 1950s, this state of the art prevailed. Solid- 
state electronics had not been applied to aircraft. Autopilots were still large, 
bulky and temperamental, but when they worked they provided the pilot with: 

— two or three axis inner loop control of aircraft heading 

— the ability to capture and maintain a magnetic heading 

— the ability to capture and maintain a VOR or ILS track 

— altitude hold and the ability to track a glide slope 

— altitude select and capture ability, in some cases. 

2.3 The Second Generation 

Turbojet transport aircraft (the de Havilland Comet in 1952 and the Boeing 707 
in 1958) provided substantial increases in speed and altitude capability, but 
required more precise inner loop control, particularly at high altitudes. Newer, 
more precise flight instruments were required by the pilots of these aircraft. 

Pallett notes that “Flight instrument evolution has followed a pattern of divergent 
display complexity with advancing technology followed by consolidation of the 
displays as human capabilities of data interpretation were exceeded.” (1983) 

Flight directors were introduced to provide command information for both the 
pilots and the automatic devices. Integrated with altitude indicators, they directed 
pilots to climb, descend and turn to new headings or radials; thus they enhanced 
inner loop control of the less stable swept-wing aircraft configuration, especially 
during low visibility and high precision maneuvers. Pilots came to rely 
increasingly on these flight directors, although substantial concerns were 
expressed at the time regarding “losing sight of the raw data” from which the 
flight director derived its information. 

The tendency of swept-wing aircraft to yaw away from banked turns (Dutch roll) 
prompted the introduction of automatic devices which counteracted this tendency 
(yaw dampers). Fast jet aircraft were similarly equipped with pitch trim 
compensators which counteracted the tendency to pitch down at high Mach 
numbers. These devices could be turned off, but in practice they were used 
without crew intervention. 

During the 1960s, advances in solid state electronics made newer, more 
competent autopilot/flight director systems possible. These systems incorporated 
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complex control laws including flare logic for automatic landings and provided a 
stepwise increase in automatic flight capability. As a further improvement, the 
automatic throttle logic integrated control of power and flight path. These 
systems were introduced in the DC- 10, L-1011, and other aircraft. 

The initial impact of these new types of automated systems led to flight crew 
reports of difficulty in learning to operate the more complex system aspects. 
Training was modified to emphasize system operation. Pilot certification began to 
require a full demonstration of the pilot’s ability to use the new systems to full 
advantage under a wide variety of circumstances where previous requirements 
had emphasized the ability to operate without such aids. 

In 1975, after a series of “controlled flight into terrain” accidents, Congress 
mandated the installation of ground proximity warning systems (GPWS) in trans- 
port aircraft. These devices used radar altimetry to evaluate height above ground 
and its rate of change; they provided light and voice wamings/commands to the 
cockpit crew when predetermined parameters were violated. Crew responses to 
the GPWS warnings were mandated by aircraft operators, but no attempt was 
made to implement these automatically (i.e., without crew intervention). Such 
systems did, however, represent a further extension of the “automated command” 
philosophy first embodied in the flight director systems, because they annunciated 
a command for the pilot to maneuver the aircraft. This was a significant exten- 
sion from the previous use of automatic devices which only maintained stable 
aerodynamic or navigational control. Today, wind shear advisory and collision 
avoidance systems are further extending this philosophy. 

In the late 1970s, the introduction of area navigation systems (first Doppler and 
then inertial) and their integration with the autopilot, added another dimension to 
the level of automation complexity in some cockpits. By the beginning of the 
1980s, airline cockpits in the operational fleet ranged from almost fully manual 
jets which had been manufactured during the 1960s to newer, highly automated 
aircraft. This mix of variously configured automation types has in itself caused 
incompatibilities. 


2.4 The Third Generation 

The next major steps in cockpit automation were the sweeping changes associated 
with the introduction of both the more flexible electronic CRT displays, the 
“glass cockpit,” and the automated system management devices. This automation 
was motivated in part by economics as well as the desire to reduce the workload 
to a level which permitted the aircraft to be safely operated with only two cockpit 
crew members. 
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Aircraft such as the DC 9-80 (now the MD-80) incorporated simplified, but 
sophisticated, aircraft management and electronic flight control systems while 
retaining the electromechanical instruments. Systems control was now affected 
through computers such as the flight management computer and alphanumeric 
control display units (CDUs) used for data entry. 

About this same time, Boeing introduced the 757/767 aircraft which used color 
cathode ray tubes (CRTs) for primary flight information and engine and aircraft 
system data. The format of the primary displays, however, retained the 
conventional, electro-mechanical presentation. One exception was a new and highly 
capable map display format which presented an effective alternative to the horizontal 
situation indicators that had been used since the early 1960s. 

These systems, along with the slightly later introduction and retrofit of new flight 
and performance management systems, completed a major revolution in the 
cockpit. Yet the implications of this new technology were only beginning to be 
understood. While manual flying was still possible, it was not encouraged or even 
practical under some circumstances. Pilots were evaluated largely on their ability 
to use fully the vastly more capable integrated flight path and aircraft 
management systems. Operationally, the new systems enabled vertical as well as 
horizontal automated navigation and guidance. Under normal conditions, thrust 
management as well had become almost completely automatic. In the final 
evaluation, pilot and crew workload for routine situations was considerably 
reduced. Other implications were, however, yet to be explored. 

The new systems provided pilots with the capability to increase flight path 
precision, and equally important, to do so with maximum fuel economy. They 
enabled fully automatic routine flight virtually from takeoff to rollout, regardless 
of weather while reducing inflight workload. 

Unfortunately, it also became evident that the air traffic management system was 
not sufficiently adaptive to permit full use of the capabilities of the newer aircraft 
flight management systems. 

Changes in ATC instructions required cumbersome reprogramming of flight paths 
through the CDUs, a task requiring considerable attention inside the cockpit 
perhaps during the descent or approach phase when attention outside the cockpit 
was most crucial. Such changes actually increased crew workload substantially and 
diverted them from managing and monitoring their aircraft’s progress and 
environment. It has been said, by both human factors researchers and operational 
experts, that the new devices tended to increase the workload when it was highest 
(climbs, descents and approaches) while decreasing it when it was already boringly 
low (cruise and high altitude flight). 
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2.5 The Next Generation 


The next generation of aircraft about to enter service (747-400, MD-11, etc.) 
have continued to introduce new forms of automation which attempt to alleviate 
some of the issues previously cited. However, this automation has also brought 
with it new forms of potential error. Some examples are discussed here to 
illustrate important trends. 

Concern about increased workload during off-nominal operations and the 
consequent diversion of crew attention from outside to inside the cockpit 
(especially at low altitude) have given rise to questions about the complexity of 
current CDUs. The newest aircraft now entering the production cycle incorporate 
attempts to simplify CDU operation by making available more comprehensive 
navigational data bases and by the addition of software to make reprogramming 
simpler. Despite these attempts, elimination of the CDU keyboard data entry has 
been seriously discussed. 

Another technological advance about to enter service is the introduction of 
advanced control systems (“fly-by-wire”) incorporating logic to prevent the 
aircraft from exceeding its safe operating envelope. Automatic limitations to 
angle of attack, bank angle as a function of air speed and configuration, and other 
control laws will insure that a pilot cannot exceed predetermined parameters 
established by the designers. These limitations apply regardless of the real time 
control inputs provided to the vehicle. 

Aircraft subsystem management will also be drastically simplified to reduce 
workload for the smaller flight crews. Microprocessor technology has been used 
to automate navigational tasks as well as management of electrical, fuel, hydraulic 
and pneumatic systems in normal operations. 

Each of these technological advances, included for a variety of reasons, has not 
only enabled new forms of errors, but also has had the effect of making the flight 
crew more peripheral to the actual operation of the aircraft (Figure 1). Whether 
this was intended or not is not within the purview of this paper, and is probably 
irrelevant. Nevertheless, the pilots who formerly exercised direct authority over 
all aspects of aircraft control and management, now have become responsible 
primarily for the management and direction of extremely complex hardware and 
software interfaces through which (and only through which) they may direct the 
operation of their vehicle. 
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Figure 1 : Evolution of Transport Aircraft Automation 
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2.6 The Historical Implications: A Summary 

Three major conclusions are apparent from the discussion of the history of the 
introduction of flight deck automation. 

• The implementation of the technology has been incremental in 
nature (a component level approach was implemented as opposed 
to a systems level design strategy). This is common with a 
technology-centered philosophy. 


The incremental development of independent, component level systems began with 
the gyroscopic stabilizer technology for aircraft attitude control. Flight control 
including fuel management became feasible and soon automated navigational 
control was enabled by the development and installation of ground based systems. 
Microprocessor and CRT technology signalled the advent of glass cockpits and 
automatic systems which directly commanded the pilot to maneuver the aircraft (i.e. 
GPWS). 

• The technology has resulted in an increasing peripheralization of 
the pilot. The flight crew has been gradually removed from direct 
control of the aircraft to control of systems which in turn control 
the aircraft. 

• Third, new types of errors have been introduced such as data 
entry errors and software engineering errors. 

Before proceeding with aviation examples, it is important to note that these same 
issues have been associated with the introduction of automation in other industries 
and a brief discussion of the implications is instructive. 

3.0 Theoretical Effects of Introducing Automation: 

Examples From Non-Aviation Industries 

The effects of advanced technology have been widely studied in other industries 
and a wide body of literature regarding automation’s theoretical basis and 
practical application has been collected. Potential solutions to problems with 
automation in the aviation domain may be placed in perspective based on a review 
of the broader experience with these technologies. 

A basic challenge to the implementation of any new automation capability is to 
predict accurately the effects of the introduced changes on other system elements. 
New automation has a large range of possible effects and ramifications that go 
well beyond the specific task addressed by the new technology. The problem is 
how to achieve the potential benefits from the new technology while finding and 
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correcting deficiencies in its utilization. Careful examination of the history of 
automation reveals that shifts in automation have system-level effects which may 
create new difficulties because the entire human-machine system has been 
changed in unforeseen ways (e.g., Noble, 1984; Hirschhom, 1984; Adler, 1986). 


3.1 Implications of Technology Centered Development 

One assumption often made is that automation will not only reduce the number of 
workers but also reduce or even eliminate worker skill requirements. However, 
actual studies indicate that this is not always the case. Automation changes the 
human role and this, in turn, changes the required skills. These skills are 
frequently more demanding. For example, more diagnostic and fault-finding 
tasks occur (Miller & Bereiter, 1986; Bereiter & Miller, 1986). 

Examples of unintended and unforeseen negative consequences that have followed 
purely technology-driven design and implementation of new automation 
capabilities include: 

A failure occurred during the introduction of a computer based power plant control 
room alarm system because the fault management information associated with the 
older annunciator alarm systems was simply no longer available (Pope, 1978; Kragt 
& Boten, 1983). 

Shifts from paper based procedures to computerized procedures have collapsed due 
to unforeseen disorientation problems by the human operator who exhibited 
different information needs with the new environment (Elm & Woods, 1985). 

These examples illustrate that increasing levels of automation create the need for 
new kinds of feedback regarding the system status, its processes, and its control 
functions. Implications for the automation designer are substantial and will be 
discussed in a later section. 


3.2 Peripheralization 

Adler (1986) describes the process of “peripheralization” that accompanies 
increased levels of automation where the human’s role is shifted away from direct 
contact with the process and becomes one of system manager, system maintainer, 
and/or machine prosthesis (provide eyes and hands for the system). Human 
peripheralization can produce the boredom/panic syndrome: 

Boredom — Situations which are within the range of competence (performance 
envelope) of the automation and there is little for the operator to do; 

Panic — Situations which approach or exceed the limits of the automation and the 
operator is suddenly thrown into a highly active, crucial role. 
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Problems have been noted in domain environments where there are extremely 
negative consequences to relatively rare events (e.g, in nuclear power plant 
settings). 


3.3 Introduction of New Error Forms 

The human’s role, by default, is to make up for shortfalls in the automated 
system’s ability to cope with the level of variety demanded by the operational 
environment (Roth, Bennett, & Woods, 1987). But, in contrast to this, the usual 
design principle is to attempt to anticipate all situations with the implication that 
designs which do not anticipate everything are bad. However, in any complex 
environment, including the National Airspace System, it is clearly impossible to 
anticipate all situations. Instead, designs should be valued for being robust in the 
face of violations of the design assumptions. 

A new form of error can also result from the tendency of automation to increase 
(or explode) the number of alerting signals that an operator must monitor. This 
has several implications: 

• Designing a supervisory control role involves the design of how the attention of 
the human operator should be distributed in different contexts and states. 

• Designs of the human interface to automated systems should facilitate proper 
distribution of attention and not degrade performance or lead to a loss of 
situational awareness. 

• There is a need to know how to design environments to support effective 
human monitoring. 


Shifts in the level of automation can effect the type, consequences and frequency 
of error occurrence. Multiple, interacting factors can produce special situations. 
Breakdowns can occur due to ambiguous instructions, special conditions or 
contexts, or erroneous assumptions about the expected operation. Frequently the 
only way to control automatics is by controlling the input. The pilot or operator 
may need to work around the automatics or may enter erroneous input with 
possible negative consequences and/or unusual system responses. 

Novel errors may be introduced when a change in one system element impacts 
other elements in unexpected ways. This may occur because automation usually 
increases the degree of coupling within the entire system. System coupling refers 
to the interdependence of subsystem elements. Error forms associated with highly 
coupled systems will increase, particularly when failures produced by several 
small, seemingly unrelated factors occur at the wrong time. For example, a 
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momentary power failure or surge can result in unintentionally reinitializing the 
system, leading to a misperception of the current system status. 

4.0 Effects of Automation Technology on Aviation 

Automation has both delivered on old promises and introduced new problems. 
Safety of flight has clearly been improved and the ability to fly in all weather and 
manner of flight conditions has been an improvement to the safety, cost 
effectiveness and utility of aviation commerce. 

Cockpit crew size has been reduced by one-third, without the imposition of 
unacceptable levels of workload on the remaining two crew members; the 
economic payoff of this change has been considerable. Area or point-to-point 
navigation, now commonplace, has shortened flight times and decreased fuel 
consumption without detrimental effects on the air traffic management system. 
The availability of automatic landing guidance has improved flight reliability, 
particularly in regional areas prone to severe weather. Flight crews, in general, 
give high marks to the automated systems in the newest aircraft in service. 

As is always the case with the introduction of new technology, automation has had 
unanticipated effects as well. However, specifics of these effects are not well 
understood, primarily because there is a paucity of quantifiable data. But, some 
of the major issues are articulated in the National Plan to Enhance Aviation 
■Safety Through Human Factors . Potential issues arising around the new, 

automated aircraft include: 

• Substantially increased head down time. 

• Difficulty in recovering from an automation failure. 

• Reluctance of flight crews to take over from a malfunctioning automated 
system. 

• Degradation of pilot skills. 

• Introduction of unanticipated failure modes. 

• Difficulty in detecting system errors. . . A __ .... 

• Incompatibility between advanced automated aircraft, existing ATC capability 
and the rest of the fleet. (ATA, 1988) 

This has necessitated a reevaluation of the implications of this technology 
particularly with respect to the current operation and future design of transport 
category flight deck automation. 

A complete account of the effects of advances in aviation automation technology 
is beyond the scope of this paper. However, a variety of problems, often only 
apparent after the new technology has been introduced and is being operated by 
real operators in real situations, have been documented in civil aviation. 
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4.1 Problems with Technology Centered Approaches 

The impact of the technology centered approach is difficult to quantify but is 
readily apparent in examining occurrences of accidents/incidents with the newer 
technology aircraft. Systems level effects pervade most of these examples and this 
category is therefore not specifically discussed. 

Examples related to peripheralization of the pilot are: 

1 . Loss of Situation Awareness : Loss of situation awareness occurs when the pilot 
develops, and fails to detect, an erroneous perception of the state of the vehicle 
and its relationship to the world. As an example, a number of authors have 
suggested that the pilot of flight KAL 007 may have: 

a) mis-set his flight management computer, 

b) failed to detect this error, 

c) remained unaware that this error would result in eventual positioning of 
the aircraft in terminally hostile airspace. 

2. Loss of System Awareness: Loss of system awareness occurs when operators of 
complex systems are fundamentally unaware of automated system capabilities and 
limitations or develop erroneous ideas of how the system performs in particular 
situations. For example, the flight crew of the China Air 747 which entered (and 
recovered from) a serious stall at about 41,000 feet over the Pacific Ocean 
several years ago was apparently unaware that: 

a) the autopilot was progressively increasing angle of attack in a futile 
attempt to maintain constant altitude when the aircraft is not capable of 
maintaining altitude on only 3 engines at 41000 feet (because of failure 
of an outboard engine to spool up following engine idle conditions, also 
produced by an automatic system). 

b) the automatic yaw damping system was incrementally increasing aileron 
control power to compensate for asymmetric yaw conditions (or 
unaware, functionally, that the automation would eventually reach limits 
in the ability to perform such compensation). 


3. Poor Interface Design: Although in some instances it is difficult practically to 
separate the interface from the automation, numerous problems have been 
attributed to poor interface design. 

For example, reprogramming the flight management system to accommodate a 
change in assigned runway during the approach phase often represents such high 
head down workload levels that pilots are unable to rely on the automated system 
to guide this phase of flight. The automated systems under these circumstances can 
be turned off and a manual approach is flown. 
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Thus, even though the autopilot fundamentally has the capability to adapt to a 
change in clearance, the actual interface with the system is so complex and time 
consuming that its usefulness is limited when it could be most effective. Good 
interface design is regrettably still an art whose successful practice is limited by 
the experience and ability of the designer. 

4. Reversion to Manual Control : Pilots of advanced technology aircraft 
understandably fear the loss of basic flight skills and many manually fly their 
aircraft in order to maintain this skill (Wiener, 1988; Curry, 1985). While it is 
difficult to find documented accidents/incidents where skill erosion has been cited 
as a contributing factor, it could be argued that the reluctance of flight crews to 
take over from automatic systems when there are good indications that something 
is wrong results in part from skill erosion. 

5. Automation Induced Crew Coordination Changes : Foushee (1988) has 
described anecdotal evidence of a particularly subtle effect of highly automated 
environments. Because much of previously observable human behavior has been 
replaced by hidden (or hard to observe) machine behavior, one can argue for the 
importance of increased (and improved) crew communication. However, at least 
one major study of crew behavior in advanced technology aircraft suggests that 
crew members may actually do just the opposite. More data are needed to 
evaluate this claim. 

Those issues related to the introduction of new error forms are: 

1. Systematic Human Procedural Errors. In a recent industry study of accident 
causation (Sears, 1986), it was asserted that fully 35% of civil accidents may be 
attributed to procedural mistakes by the flight crew. While the reasons for such 
lapses are complex and incompletely understood, as an approximation they can be 
attributed to human limitations associated with working memory, with 
inappropriate attention allocation, with information processing limitations under 
stress of various types (e.g., time, workload, etc.), and with human tendencies to 
make performance “slips,” which have been documented most recently by 
Norman (1981) and Reason (1987). 

2. Systematic Decision Errors : A group of psychologists and cognitive scientists 
collectively known as “Subjective Decision Theorists” has, over the past two 
decades, done a revealing job of cataloging systematic biases which limit the 
ability of humans to make optimal decisions. Woods (1988), and others have also 
shown that joint or cascaded machine-human decision systems often exhibit 
suboptimal performance, relative to what might reasonably be expected on the 
basis of individual competencies. In the aviation context, the Air Florida Flight 
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90 accident at National Airport has been asserted (Nagel, 1988) to illustrate a 
variety of classical human decision limitations. 

5.0 Technology Centered versus Human Centered Automation 

We have argued, although it is somewhat difficult to find explicit supporting data, 
that the introduction of automation has been technology driven. That is, most 
automation technology has been implemented without a clear understanding or 
perhaps explicit statement of the actual problem that the technology is supposed to 
solve. A recent industry report states that: 

“There are many problems associated with the introduction of advanced technology 
onto the flight deck of transport category aircraft. Many of these arise from the lack 
of a consistent ‘philosophy of automation’. To date, the designer has largely not 
been constrained by human factors considerations nor guided by a global approach 
to the introduction of automated systems and procedures. This has resulted in 
designs where the crew has been forced into the system almost as an afterthought 
and frequently is outside the system control loops. It also results in operations in 
which the human is primarily a monitor of the automated systems and yet data 
indicate that humans are totally miscast in this role.” (Norman, et al., 1988) 

Such implicit automation philosophies which have guided much of advanced 
systems development in aviation and elsewhere, have tended toward 
implementations which replace human function with machine function — the 
technology centered approach. As a result, the human has not always been left 
with a task environment that is fully compatible with human capabilities and 
limitations. In this sense, the technology driven approach also has had the effect 
of leaving the human out of meaningful control and/or active participation in the 
flight operation. This “ Human-Out ” phenomenon need not necessarily be 
associated with technological advances, but unfortunately it is normally what 
happens. 

A logical contrasting philosophy to that of slowly removing the human from 
substantive task responsibilities is to design systems such that the human is the 
central element in control and management of the system — a " Human centered ” 
approach. Implicit in this approach is the development of designs which: 

• fully utilize and enhance the unique human capabilities of pattern recognition, 
information integration, learning and adaptation, etc. 

• protect the system from human limitations such as systematic human error 
tendencies, unreliable monitoring skills, decision-making biases and limitations 
of working memory and “processing speed”. 

Note that this human centered philosophy, as stated, is mute on the topic of 
whether the flight crews are allowed to choose to do anything they desire. 
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Additional operational procedures are required which, depending on the situation 
(system context), would state the rules and circumstances for use of specific 
automated capabilities. 

5.1 Toward a Working Philosophy 

It is not the purpose here to define a human-centered philosophy of automation. 
Instead we attempt to prepare the foundation for such a philosophy by describing 
its basic concepts and developing a conceptual framework for design. Toward this 
end, the realities of the aviation environment must form the basis for any 
philosophy which is expected to be actually implemented. 

Realities: 


• The civil aviation system is complex; this complexity is increasing. 

• The National Airspace System (NAS) is heterogeneous; it is a highly coupled 
system involving a mixture of humans and machines, each with a very broad 
spectrum of capabilities. 

• The flight environment is highly unpredictable (e.g., weather). Problems in one 
part of the system have effects at distant times and places (e.g., traffic delays). 

• The flight crews and air traffic controllers who operate the system, possess 
(now and for the foreseeable future) unique perceptual and cognitive abilities 
which are as yet unmatched by artificial systems. 

• These same human operators have performance limitations which are 
increasingly well known and understood in the context of predictive modelling. 
For example, humans need help with very fast or very slow event sequences. 


5.2 A Conceptual Framework 

As we have noted, a working philosophy must be able effectively to consider 
three important concepts; 1) a system level perspective, 2) peripheralization 
issues, and 3) management of new error forms. 

1) System Level Perspective: Aviation system design must reflect 
global systems engineering approaches and practices. It can no 
longer be designed and developed incrementally using a piecewise 
integration of independently designed components. 

The global questions which should be directed to the automation designer and 
operator are best summarized by the following; 
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a. Is there an explicit understanding of the implications of the in- 
troduction of the new, automated technology on the total system, 
particularly the role of the human? 

b . Have the reverberations throughout the total system been antici- 
pated? 

c. Has the new human role been properly supported? 

These questions revolve around the central issues of function allocation between 
the human and the machine. Such choices need to be decided both on the basis of 
overall system effects including the performance/cost improvements and with 
consideration for the potential of introducing new error forms. The system 
perspective is most crucial. 

2) Peripheralization: In order to counteract the effects of 
peripheralization, human-centered automation systems must be 
designed which: 

a. allow for human interaction and involvement with the system 
which is consistent with human intellectual abilities, skill level, 
and responsibility; 

b . allow for the joint and collaborative interaction and responsibili- 
ties of flight crews, controllers, and ground personnel; and 

c. enhance unique human capabilities. 

The question of providing proper support for the new role of the human after the 
introduction of new automation is important. Automation, as previously stated, 
increases the peripheralization of the human. This frequently means that the 
human role is changed from one of direct control to one of issuing coordinating 
instructions. Automation designs, however, do not always allow the operator to 
intervene. In these cases the person has to find ways to “get around” or “trick” 
the system to get it to perform correctly, particularly under special 
circumstances. This situation may have resulted from a form of “designer 
overconfidence.” However, it is difficult both to design an interface so that 
intervention is possible and to analyze all the possible circumstances which could 
require the operator to intervene. But these issues are cmcial to the successful use 
of automation and they form the basis for the design principles which follow. 

3) New Error Forms: Aviation system elements should be designed 
so that they are effectively protected from systematic human 
limitations analogous to designs which have been used to protect 
against machine failures. 
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In the system level view, the introduction of new error forms are interpreted as 
symptoms of deficiencies or flaws in the underlying automation design or 
architecture (Hollnagel and Woods, 1983). Such errors point to the areas where 
imp rovements in the architecture of the automation are needed. 

This is in stark contrast to the view that the human is an independent source of, 
or contributor to, errors. Yet the technology-centered approach often assumes 
that the pilot is an independent source of errors so that such events are often 
remedied by increasing the level of the automation. We have seen this frequently 
introduces new forms of errors. 


5.3 Emergent Principles for Design 

1. Decision Support: Joint cognitive systems should be designed to avoid 
problems associated with human/machine decision systems. Decision support 
should take the form of machine supported human decision making paradigms. 
They could be designed so as to reduce decision biases and provide both action 
sequences and their consequences. An attempt should be made to create systems 
that are both error tolerant and error evident. Warning and alerting systems 
should be: intent-driven, based on strategic goals selected by the crew; predictive, 
able to forecast critical conditions; intelligent, able to recognize inconsistent 
input; and able to give explanations if necessary. System designs should allow 
maximum discretion of the crew in decisions to employ or not to employ cockpit 
devices. 

2. Interface Design : A most critical element is the task representational aspect of 
int erface design. Available evidence suggests that cognitive/perceptual tasks are 
best mediated with information systems and displays which minimize the kind and 
type of mental “transformations” required to translate between physical and 
cognitive representations. Automated systems designers should evaluate the way 
the pilot would perform the same function. Intuitive design allows the pilot to 
understand more easily the automated system and if necessary take over from it. 
Additionally, consideration should be given to annunciate to the pilot how well 
the automatics are performing their function and how close the automatics are to 
their own performance limits. This clear annunciation of the status of the 
automation is termed transparency. In other words, the pilot should be able, 
easily and quickly to understand and check machine status and parameters. 

3. Procedures Support : Humans are poor and unreliable monitors (Wiener, 
1987). They often fail to follow established procedural protocols under actual 
operational conditions. New technologies (intent inferencing, activity tracking, 
etc.) promise systems which have the capability of “intelligently” monitoring 
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humans. Procedural aiding systems, which have been used successfully in process 
control settings, offer promise for reduction of procedural errors in aviation. In 
system design, procedural aspects should be considered so that lower priority 
pilot tasks do not interfere with the satisfactory performance of higher ones. 
Additionally, consideration should be given to reducing the time to switch 
between tasks. 

4. Crew Coordination: Foushee (1987) has pointed out the importance of 
effective communication practices between multiple crew members in highly 
automated cockpits. Formerly visible actions of human crews have in many cases 
been replaced by the invisible actions of imbedded machinery. 

5. Workload Management: The strategic management of workload consists of a 
redistribution of tasks, reducing workload where it is high or possibly increasing 
it where workload is low and the flight regime is not critical. Cockpit tasks, 
supporting materials, and ATC procedures should be designed to minimize head- 
down time during critical phases of flight. If workload is properly managed, 
situational awareness will not suffer due either to very demanding task 
requirements or low task priority. 

5.4 Emergent Principles for Operation 

A more complete treatment of this topic is required, but this brief statement is 
included to indicate that principles of operation are as (or perhaps more) 
important than those applied for design. 

Training: Initial emphasis should be on the “basic airplane” before introducing 
the automation. LOFT exercises should allow flight consistent with the principle 
of machine-supported human decision making. 

Procedures and Policy: Company policy and procedures should consider the 
impact of additional cockpit duties on high workload activities (e.g., making 
company radio calls). Careful consideration should be given to operational 
procedures particularly for crews flying derivative type airplanes, i.e., the fleet 
contains both traditional and advanced technology derivative aircraft. In these 
instances, consideration should be given to training issues as well as the 
development of procedures for the advanced technology derivative. 

6.0 Toward Implementation 

Much work needs to be completed before the emergent principles and other 
factors described here can be translated into a useful working philosophy for the 
design and operational environment. It is possible, however, to describe some of 
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the realities of implementation as well as the components of a philosophy which 
can then form the basic framework for further work. 

Current Implementation Practices: 

• A new cockpit is rarely designed from the top down. Most are incremental. 

Design limitations must be recognized and accounted for in a philosophy. 

• Currendy the only human factors issue in the certification process (part 25) is 
workload. System level effects, peripheralization issues and potential for new 
errors need to be considered. 

Components of a Philosophy: 

A philosophy of automation specifies the crucial interactive nature of the 
relationship between the human and the machine. 

Any design philosophy of automation should have the following 

characteristics: 

• Explicitly describe design and o perational principles. 

• Predictive ability to sort out improvements versus potential hazards in design 
and procedures at a gross level. 

• A procedure for analysis which will enable the consequences of design 
decisions to be described. 

• A system level, not a component level, perspective. 

• Assessment criteria which allocate functions between the human and machine 
and a process for their application. 

• An analytical test to identify inconsistencies in the application of the philosophy 
or problems in its implementation. The test environment should be a systems 
level, operationally oriented test, that would describe a non-standard but 
frequently occurring situation such as a change in assigned runway on a close in 
approach. The performance of specific data entry systems and procedures could 
then be effectively evaluated in these well defined, benchmark situations. 

A philosophy of automation is needed to provide consistency in design and 
operation. It should provide a view which is consistent and, therefore, supports 
system reliability. A philosophy provides design constraints on human/machine 
interaction and is needed to guarantee that the role of the human will not be 
unduly compromised by design or procedural expediency, cost or simply lack of 
awareness. 


185 


An automation philosophy provides guidelines and constraints for answering 
design questions, especially where experimental data are not available or not 
possible to obtain. It provides a methodology to evaluate both individual design 
decisions and the overall utility of automation technology. 

A crucial element in any philosophy is the role of the human. The difficulty is 
one of making a conscious choice to define and implement this role and to use 
technology in its support. This leads us to the last and perhaps most important 
questions surrounding aviation automation — the role of the pilot in the future 
system. 

7.0 The Role of the Human in Future Systems* 

It is not unreasonable, given the capabilities of automated aircraft about to enter 
service, to ask whether in fact the human crew could not be eliminated in routine 
operations. Given that air traffic control were to gain a higher bandwidth means 
of communicating control instructions to aircraft, would it not be possible for 
those instructions to be translated directly into commands for flight path 
management? The answer to this question, given full implementation of 
technology currently available, is an unequivocal “yes.” If the automated aircraft 
systems can become sufficiently reliable (and the newest systems have been 
designed to be very reliable indeed), there is no reason why ground-directed 
flight, from takeoff roll to landing rollout, cannot be fully automated. 

The fact that fully automated flight except for taxi and “unusual circumstances” is 
possible with at most minor improvements in current technology gives rise to the 
concern voiced by Wiener and Curry in 1980: " The question is no longer 
whether one or another function can be automated but, rather, whether it should 
be.” 

For many reasons, including reliability and passenger acceptance, it is extremely 
unlikely that unmanned, fully automatic aircraft will be introduced into air 
transport at any time in the near future. That statement, however, begs the 
question of whether manned , fully automatic aircraft should be introduced; that 
is, aircraft that require perhaps some monitoring, but no human intervention 
during normal operations, except perhaps during taxi and ground operations. 

The question posed by Wiener and Curry is very nearly academic. Almost fully 
automated aircraft are about to be introduced into air transport operations. 


* This section is reprinted from an unpublished manuscript entitled, “Toward Human Centered 
Automation”, with the permission of the author, Dr. Charles E. Billings. 
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whether they should be or not. Perhaps a more appropriate question would be 
“Given the level of automation now available in transport aircraft, what should be 
the role of the pilots?” 

Automation could be designed to keep the pilot closer to the control of the 
vehicle, while providing an array of information management and aiding 
functions designed to provide the pilot with data regarding flight replanning, 
degraded system operation, and the operational status and limits of the airplane, 
its systems and the physical and operational environment. Outer loop functions, 
including monitoring of operator performance, could be components of such a 
philosophy of automation. The pilot could call on automation modules to assist in 
problem diagnosis, in evaluation of available alternatives, and in execution of 
alternative plans. The automation would serve as the pilot’s assistant, providing 
and calculating data, watching for the unexpected, and keeping track of resources 
and their rate of expenditure. 

Is such “human-centered automation” possible? The answer is certainly “yes.” Is 
it likely? Unfortunately, it is exceeding unlikely unless serious thought is given to 
the direction of our past and current automation development, and unless a new 
automation philosophy is adopted prior to the design of the next generation of 
transport aircraft. 

We do not suggest that it is a conscious aim of the designers of current transport 
aircraft to eliminate the flight crew from a central role in aircraft and aviation 
system management. The direction of the trend in automation technology, 
however, may well have precisely that result. If this is not to happen, we may 
have gone as far or farther than we should go without making explicit our 
philosophy of automation and examining the directions in which our automation 
technology development are leading us. 
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APPENDIX C 


INSTRUCTIONS TO WORKING GROUPS 

The most important elements of the workshop on Flight Deck Automation: 
Promises and Realties are the discussions which will occur in the small working 
groups. Each participant has been assigned to a specific working group. These m- 
depth discussions are critical because they provide the basis for the final products of 

the workshop. 

The panels and invited papers were selected to prepare a foundation for discussion 
by describing ideas, concepts and terminology. The application of these concepts 
and ideas will hopefully lead to new, more creative ways of understanding the issues 
associated with flight deck automation. Since the intent of the workshop is also to be 
of practical value, identification of current approaches or coping strategies used 
with the current generation of aircraft is also important. 

Although the “Philosophy of Automation” is a major topic, the conference is not 
intended to be theoretical in nature. It is, however, important to understand and 
assess the existing situation before any changes, future research programs or 
philosophies are developed. Obviously a view toward the future is important and 
should be included in the discussion, but a critical, exact understanding o t e 
current situation must form the basis for any discussions of the future. 

The time allocated to these working groups by each of the participants is very 
valuable and it is a rare opportunity for a broad representation of the aviation 
community to be able to come together to discuss complex issues. 

Because of the importance of these working groups, considerable time and effort 
has been taken in the development of their goals and objectives. Careful 
consideration has been given to developing workable, productive objectives, lne 
information attached has been prepared to initiate discussion and to assist the 
working group chairs, the NASA vice chairs and the individual participants. 

Objectives 

• Identification of the issues regarding flight deck automation 

• Prioritization of these issues 
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• Identification of coping strategies used in current operations 
training, and ATC interface. 

• Identification of design and operational guidelines. 

Expected Products of the Working Groups 

1. An informal, interim verbal report for Wednesday mo rnin g, and 
a final verbal report for the closing session on Thursday. 

2. A final written report will be prepared (format is described 
below). 

3. A list of terms which serve to describe automation related issues 
(an initial draft has been prepared by Ev Palmer). 

Procedures: 

Each working group has an assigned chair from the industry, and a NASA vice 
chair who will serve multiple functions, including host, resource person, 
technical and scientific advisor, and recording secretary. Detailed working 
procedures have been left to the discretion of each chair. 

Working group assignments were made by NASA on the basis of several 
considerations. Please note that although we have assigned one of six specific 
areas to each working group, we do not mean to limit the scope of any group’s 
discussion to the assigned area. Instead, consider the assigned area to be a focus 
for the primary area of consideration. If you wish to make recommendations in 
other areas, you certainly may do so after a thorough development of the 
primary area. 

The working group’s discussions are intended to be informal. The idea is to 
describe the benefits and issues associated with flight deck automation and to 
prioritize their importance. This prioritization should (where applicable) rank the 
issues by their frequency of occurrence (i. e. the most common to the least 
common). In addition, areas with a low frequency of occurrence, but high 
consequence, should be identified. Other factors such as ease of change also may 
be considered. It is important to note that a consensus at this point is not 
necessary. 

Please note that two NASA participants, Susan Norman and Michael Shafto, do 
not have working group assignments. They will circulate among all the groups 
throughout their discussions. 
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Working Group Reports 

It is the responsibility of your NASA vice chair to help the chair draft the 
working group report which will be submitted to the general assembly. To insure 
a uniform format, we ask that these reports follow the format suggested below. 

Each report should consist of three major sections: (1) an Introduction; (2) 
Recommendations; and (3) a Summary. The Introduction should describe general 
features of the approach taken to the assigned issue by the working group, and 
other materials of a general, introductory nature. 

The Recommendations section is the heart of the report. This section should 
directly address three topics: (1) Issues Identified and Prioritization of Issues; (2) 
Guidelines and Current Coping Strategies; and (3) Recommendations for the 
Future. 

Any major areas of disagreement, minority opinions, and other similar 
information should be placed in the summary section. 

The interim status report is intended to be informal. It should be a short 
verbal report of about 10 minutes and is intended to foster discussion among the 
other working groups. 

The final report should be written (viewgraph or text ) but will be presented 
verbally on Thursday morning. Resources such as Macintosh computers are 

available. 

We realize that the issues described here are complex and that a full description 
of each issue may not be possible or desirable. Our intent is to provide a basis for 
understanding how these issues relate to one another and not necessarily to 
understand any one issue completely. With this in mind, it is more important to 
focus on the broad, interactive aspect of automation than to fully describe specific 
elements 

Thank you for your participation in this workshop. It is through your efforts that 
we can obtain a better understanding of the complex issues surrounding flight 
deck automation. 
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Flight Deck Automation: Promises and Realities 
Working Group 
CREW COORDINATION 


Chair: A1 Ogden 

Vice Chair: Barbara Kanki 

The design and implementation of increasingly automated systems on the flight 
deck brings raises a variety of potential human factors issues relating to 
individual crewmembers. In addition to these concerns, however, there are issues 
which may affect the crew as a whole; that is, the way in which crewmembers 
coordinate their activities together. The most obvious, direct effect include 
changes in task structure, changes in information flow and communication 
channels, and changes in the interpersonal aspects of traditional and standard 
procedures. 

There are also indirect effects (i.e., effects which are less specifically tied to 
flying the aircraft). These include changes in the organizational structure of the 
crew such as shifts in authority and responsibility as well as effects related to the 
problems of transition from one technology to another and training. 

I. Direct effects of flight deck automation on crew coordination 

A. Changes in task structure 

i. type of tasks, e.g., active vs. monitoring 

ii. distribution of workload, “who does w ha t” 

B. Changes in information flow/communication channels 

i. face-to-face information transfer may decrease 

ii. information may be maximally available but “who knows what” 
no longer public knowledge 

iii. changes in ground-based support role 

C. Changes in interpersonal aspects of standard procedures 

II. Indirect effects of automation on crew coordination 

A. Social/organization issues 

i. who is in authority/who holds the information 

ii. who is responsible for shared info 

B. Effects of transition 

i. must pilots be fluent in both systems for backup 

ii. what are the rules for switching (an additional 
interface task) 

iii. training must incorporate changes 
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UNDERSTANDING AUTOMATED SYSTEMS 

Chair: R. Braune 

Vice-Chair: A. Lee 

The purpose of this working group is to provide a forum for an interdisciplinary 
discussion on the need for, and implications of, operator understanding of 
automated systems. Automated system is broadly defined as any self-operating 
machinery which controls or performs tasks (e.g., an FMC). Issues included in 
this group discussion is the extent to which operators need to know the operating 
principles of automated systems and the need for maintenance of operator 
awareness of the status of such systems. The extent to which operators require 
explanations of system actions and the need for operators to anticipate actions of 
the system will be addressed, finally, the implications of these and other related 
issues on operator training, a system design, and operating procedures will also 
be discussed. 
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WORKING GROUP: AIR-GROUND 
COMMUNICATIONS (ATC) OBJECTIVES 


Chair: Jack Wisely 

Vice Chair: Renate Roske-Hofstrand 

Among some of the issues to be discussed in this working group are how 
increased cockpit automation affects the pilot’s interaction with ATC. We should 
discuss both negative and positive effects. Possible sample issues include the 
following: 

1 . Air-Ground Matching (Mis-matching) 

2. Flight Plan Changes and CDU re-programming 

3. Cooperative Behavior (Responsibility and Reliance) 

4. Pilot perceptions regarding new control procedures such as: 

• Flow Management 

• ATC intervention only to prevent conflicts 

• Communications Management 

• Enroute delays 

If we were tasked to establish guidelines for designers of cockpit automation what 
would we have to say? 

What should we know about the FAA’s Advanced Automation System that could 
or should influence the design of on-board automation tools? 

To stimulate our discussions, I have attached a brief article entitled “The Quest 
for Airspace Safety and Capacity” which reports on the UK’s National Air 
Traffic Services’ (NATS) attempt to deal with increased demands. 
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Flight Deck Automation: Promises and Realities 
Working Group: Error Management Issues 


Chair: David Nagel 

Vice Chair: Everett Palmer 

The objective of this working group is to identify the influences, both positive 
and negative, of cockpit automation on the occurrence and detection of error on 

the flight deck. 

A key goal in the design of aircraft cockpits, aircraft operating procedures and 
crew training is the reduction of incidents and accidents attributed to human 
error. Some have claimed that automation can eliminate human error by 
removing the pilot from the control loop. Others have claimed that while some 
types of errors may be reduced the automatic equipment itself introduces 
opportunities for new types of human error. Like the digital watch it may 
eliminate small errors but introduce the possibility of large errors. These new 
error forms seem to be particularly difficult to anticipate at design time. 

We will discuss: 

• The changes in cockpit systems that have affected the type and 
frequency of crew errors. 

• Examples of types of human error that have been reduced. 

• Examples of new types of human error. 

• Methods for anticipating the impact of new technology on human 
behavior and on the nature of human errors. 

The key output of this working group will be a prioritized list of automation 
issues and how they relate to errors and error detection in advanced technology 
cockpits. 
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Flight Deck Automation: Promises and Realities 

Working Group 

Training and Operational Procedures 

Chair: Frank Tullo 

Vice-Chair: Harry Orlady 

Operating procedures and basic training curricula are developed by the 
manufacturer in order to operate their aircraft safely and efficiently under all of 
the conditions they may be exposed to in transport operations — i.e., standard 
operating conditions, abnormal conditions, and emergencies. They are then, with 
the approval of the airline’s FA A Principal Inspector (PI), frequently modified to 
be consistent with the general operating practices of the airline. After the 
procedures have been developed and approved pilots must be trained to use them. 

Pilot training for all airlines is both important and very expensive. While there is 
no disagreement regarding its importance, there has not always been agreement 
on the kind and amount of training that is required to enable pilots to operate new 
and different airplanes safely and efficiently. As an entirely separate issue, there 
is also controversy regarding the effect of automation on training requirements. 

The training issues are complex. They can vary between aircraft and among 
airlines and over time. They can be identified and categorized in many ways. The 
following issues and sub-issues have been discussed at considerable length in the 
literature often with considerable disagreement. Here they have been divided 
into four entirely arbitrary categories: 1) Conceptual or Generic Issues; 2) 
Implementation, Company Policy, and Procedural Issues; 3) Transition Training 
Issues; and 4) Recurrent Training Issues. There is considerable overlap among 
the categories. & 

I. Conceptual or Generic Issues 

- The “Changing Role” of the Pilot 

- Effective Crew Coordination in ADVTECH Aircraft 

- Monitoring and Vigilance in ADVTECH Aircraft 

- Understanding System (including Software) Logic 

- Low-time Pilots in ADVTECH Aircraft 

- Cabin Crew/Cockpit Crew Coordination 

- Ab Initio Training for ADVTECH Aircraft 

- Instructor Training for ADVTECH Aircraft 

- Short vs. Long-haul Operations 
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II. Implementation, Company Policies, and Procedures 

- Utilization of Advanced Technology 

- Maintenance of Manual and Cognitive Skills 

- Emphasis on “Automatics” vs. Basics 

- Heads-Down Time in Traffic 

- Contracted Training 

III. Transition Training Issues 

- Adequacy of Transition Training Program 

- Sensitivity to Varying Pilot Training Needs 

- Sensitivity to Line Operating Needs 

- Appendix E Maneuvers and Procedures 

- Initial Operating Experience 
“Differences” Training with “Common Type” 

- Computer-Aided Instruction for ADVTECH Aircraft 

- Effectiveness of Flight Management System Trainers 

- Specific Transition Training Issues 

• Transition to EFIS 

• Aircraft Systems 

• The Autopilot/Autothrottle, FMS, and MCP 

IV. Recurrent Training Issues 

- Adequacy of Recurrent Training Programs 

- CRM (Cockpit Resource Management) for ADVTECH 
Aircraft 

- LOFT (Line Oriented Flight Training for ADVTECH 
Aircraft 

- Appendix F Maneuvers and Procedures 
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APPENDIX D 


AUTOMATION TERMS AND ACRONYMS 


TERMS 

Note: Terms in italics are defined in this appendix. 

actual complexity - The actual complexity of the system is generally of interest 
only to the designer and the maintenance personnel. It has to do with how the 
system actually functions. 

automated aircraft or ADVTECH aircraft - Advanced technology aircraft 
with CRT displays and Flight Management Systems, such as the Boeing id / & 
767, 737-400, 747-400, MD-88, and Airbus A-310, A-320. 

automatic - “acting or operating in a manner essentially independent of external 
control; self-regulating” — but also, “lacking volition, intention, or conscious 
planning” (American Heritage Dictionary, 1976). 

automation - a. “automatic operation or control of a process, equipment, or a 
system” American Heritage Dictionary, 1976) b. any computer processing ot 
information displayed to the pilot or of control inputs from the pilot, c. using a 
computer to accomplish a task that was or could be done by the pilot, d. using a 
computer to make decisions that were previously made by the pilot. 

backgrounded tasks - Tasks that have been delegated to another agent, either 
machine or human. A key factor is that the person who delegates the task 
remains responsible for its successful accomplishment. 

breakdown - An automated system is in breakdown when its current operational 
environment was not anticipated by the designers of the automated system and it 
cannot cope with the environment to achieve its goals. 

breakdown of the system image - The actual behavior of the system is 
inconsistent with the system image. This could be due to problems in the design 
of the system or due to hardware or software failures. 

brittleness - A characteristic of some forms of automation if they cannot cope 
with unanticipated situations. The system performs correctly only for situations 
that were anticipated during the systems design. 
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bumpless transfer of control - Smooth transfer of control between manual 
and automatic control modes; for example, if an autopilot is disengaged while 
it is holding full right rudder, the transfer to manual control is “bumpy” if the 
rudder correction is suddenly removed. 

clumsy automation - Automatic systems equipment which are difficult to use 
and understand. It may perform the necessary functions but is difficult to use. 

complacency - A false sense of security induced by the reliable functioning of 
automatic equipment. Failing to maintain situation awareness because of a 
reliance on automatic equipment. 

coupling - A system is highly coupled when a failure of one component of the 
system affects the functioning of other system components. These effects occur 
quickly. 

Error Terms 

slip - An error in which the person has the correct intent but errs in 
executing the action. 

mistake - An error in which the person forms the wrong intent. 

This wrong intent may be executed correctly. 

error displacement - a. Errors made by the system designer 
which are not evident until the system is in operation, b. The 
consequences of an operator error is not evident at the time the 
error is made. The error only becomes apparent later in the 
flight. 

error evident - The system is designed so that errors are detectable 
by the operator in a reasonable time period. 

error propagation - Errors in one part of the system affect the 
functioning of another part of the system. 

error tolerant - Errors are detected by the system and annunciated 
before their consequences degrade system performance. 

fixation - a. A human operator makes a situation assessment and 
then fails to change it in the face of new evidence that is contrary 
to the initial situation assessment, b. Cognitive hysteresis. 
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envelope protection - Does not allow any pilot inputs which exceed the 
performance envelope of the aircraft. An example is alpha floor or alpha 
limit.” The pilot cannot increase the pitch angle to the stall angle-of-attack. 
This is possible on fly-by-wire control systems. 

event oriented procedures - Procedures that attempt to locate the event which 
is causing a system malfunction. 

function oriented procedures - Procedures that attempt to maintain and 
restore the critical safety functions of a system. They are not necessarily 
concerned with diagnosing the event which caused the system malfunction. 

glass cockpits - Advanced technology flight decks that use CRT (cathode ray 
tube) (glass) displays for the primary flight displays. 

mediated - a. Information to the pilot and controls from the pilot are processed 
by a computer, b. The user works through a computer to accomplish a task. 

Model Terms 

conceptual model - The users’ model of how the system works. 

The user can form expectations about the behavior of the system 
in new situations based on his or her conceptual model of the 
system. 

design model - The designers’ model of how the system works. 

This is the system model that the designer would like the user to 
have. It is also how the designer hopes the system will actually 
operate. 

system image - The image the system presents to the user. The 
designer should attempt to make the system capabilities and 
operations evident from this image. The system image includes 
the design of all of the visual and interface parts of the system. 

user model - This is the model of the system that the user actually 
develops. It can be based on the system image, training materials, 
knowledge of task or knowledge of other similar system. 

perceived complexity - The users’ view of the complexity of the automatic 
system. This is the complexity of the “user model.” 
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performance envelope - The boundary conditions within which a system can 
perform correctly. For example, aircraft performance envelopes define power 
and altitude limits as a function of aircraft configuration. Conceptually, 
automated systems also have performance envelopes, but they are often not 
well understood by system operators. 

redundancy - More than one independent method exists for accomplishing a 
function or task. 

reversion - The process of switching from automatic to manual control. 

supervisory control - The supervisor is responsible for successful completion 
of the mission. Low level tasks are delegated to other machines or human 
agents. Generally the automatic equipment exercises the “manual” control, 
while the human sets goals and monitors the system as a whole. 

transparency - a. A computer system is transparent if the user feels that he/she 
is operating directly on the task and not using a computer to accomplish the 
task, (example: many Macintosh applications.) b. A computer system is 
transparent if it is clear how it operates. 


208 



ACRONYMS 


AAS 

ACARS 

ADF 

AFS 

AP 

APU 

ASRS 

ATC 

Basic “T” 

C/W 
CAT II 

CATIH 

CDU 

CM1 

CM2 

CRT 

CWS 

ECAM 

EEC 


Advanced Automation System for Air Traffic Control 
Automatic Communication and Recording System 
Automatic Direction Finder 
Autoflight System 
Autopilot 

Auxiliary Power Unit 
Aviation Safety Reporting System 
Air Traffic Control 

Traditional spatial configuration for display of primary flight 
instruments. 

Caution/Waming 

Category II approaches have weather minimums of a 200 ft ceiling and 
2600 ft RVR. 

Category III approaches have weather minimums of RVR 700 ft and 
no minimum ceiling for landing. Subcategories of CAT III have even 
lower minimums. 

Control Display Unit - the pilot’s interface with the FMC 

Captain 

Copilot 

Cathode Ray Tube 
Control Wheel Steering 
Electronic Centralized Aircraft Monitor 
Electronic Engine Control 
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EFIS 


Electronic Flight Instrument System 
Exhaust Gas Temperature 


EGT 

EICAS Engine Indicating and Crew Alerting System 

F/D Flight Director 

FMA Flight Mode Annunciator 

FMC Right Management Computer 

FMS Right Management System 

G/A Go-around 

GPWS Ground Proximity Warning System 

ILS Instrument Landing System 

INS Inertial Navigation System 

IRS Inertial Reference System 

IRU Inertial Reference Unit 

LNAV Lateral Navigation 

LRU Line Replaceable Unit 

MCP Mode Control Panel 

MODE S: When implemented, Mode S will provide the medium for a digital data 

link which will be used to exchange information between aircraft and 
various ATC functions and weather bases. 

NAS National Airspace System 

NDB Non-Directional Beacon 

OMEGA Enroute long-range radio navaid of very low frequency (VLF) 
hyperbolic type 
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PF 


Pilot-Hying 
PNF Pilot-Not-Flying 

PROF Profile - used in vertical navigation 

RVR Runway Visual Range 

TCAS Traffic Alert and Collision Avoidance System 

V/S Vertical Speed 

VHF Very High Frequency 

VNAV Vertical Navigation 

VOR Very High Frequency (VHF) Omnidirectional Radio Range 
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